Content area
Purpose
This study aims to provide measurable information that evaluates a company’s ESG performance based on the conceptual connection between ESG, non-financial elements of a company and the UN Sustainable Development Goals (SDGs) for resolving global issues.
Design/methodology/approach
A novel data processing method based on the BERT is presented and applied to analyze the changes and characteristics of SDG-related ESG texts from companies’ disclosures over the past decade. Specifically, ESG-related sentences are extracted from 93,277 Form 10-K filings disclosed between 2010 and 2022 and the similarity between these extracted sentences and SDGs statements is calculated through sentence transformers. A classifier is created by fine-tuning FinBERT, a financial domain-specific pre-trained language model, to classify the sentences into eight ESG classes.
Findings
The quantified results obtained from the classifier reveal several implications. First, it is observed that the trend of SDG-related ESG sentences shows a slow and steady increase over the past decade. Second, large-cap companies relatively have a greater amount of SDG-related ESG disclosures than small-cap companies. Third, significant events such as the COVID-19 pandemic greatly impact the changes in disclosure content.
Originality/value
This study presents a novel approach to textual analysis using neural network-based language models such as BERT. The results of this study provide meaningful information and insights for investors in socially responsible investment and sustainable investment and suggest that corporations need a long-term plan regarding ESG disclosures.
1. Introduction
As environmental, social and governance (ESG) considerations have gained significant global trends, the management and implementation of ESG have become a considerable concern for both corporate entities and investors alike (Amel-Zadeh and Serafeim, 2018; Kotsantonis et al., 2016). To enhance investment attractiveness, companies are increasingly disclosing their ESG performance (Saad and Strauss, 2020); however, they are faced with the obstacle of the absence of standardized ESG metrics (World Economic Forum, 2022). In response to this challenge, some companies have turned to utilizing the Sustainable Development Goals (SDGs) adopted by the United Nations (UN) in September 2015 as a guide for ESG disclosure standards (Blasco et al., 2018). It is recognized that ESG and SDGs are interconnected, and the pragmatic framework provided by SDGs serves as a possible starting point for companies to implement ESG (Berenberg, 2018). Consequently, an analysis of the relationship between these two concepts in business activities can aid in explaining a company’s ESG performance (Amel-Zadeh et al., 2021). Nonetheless, measuring a company’s ESG alignment with the SDGs and determining the extent of the company’s contribution to each SDG are not a straightforward task (Khaled et al., 2021). In light of these challenges, a primary objective of this study is to quantitatively assess SDG-related ESG disclosures and furnish information that can be utilized to evaluate corporate ESG performance, thereby supporting investment decision-making.
Companies’ disclosures, corporate social responsibility (CSR) reports and sustainability reports are publicly available source that allows investors to verify a company’s ESG activities and performance. In particular, Form 10-K, an annual report submitted by public companies to the U.S. Securities and Exchange Commission (SEC), is considered a valuable source of data for business analysis (Li, 2010; Yuthas et al., 2002). It contains a comprehensive overview of a company’s annual management activities, including both qualitative data, such as industry conditions, forecasts and investment plans, and quantitative data, such as financial statements. Some U.S. public companies have been voluntarily disclosing information about their non-financial management activities through Form 10-K even before the increased interest in ESG and SDGs. Form 10-K has been used as the primary raw data in various studies that analyzed non-financial activities of companies, such as climate change risk (Doran and Quinn, 2008), gender equity (Nadeem, 2022) and accountability (Enache and Hussainey, 2020).
Form 10-K composes a vast amount of unstructured text, including numeric data and characters. Therefore, specific data processing procedures and techniques are required for its analysis (Loughran and McDonald, 2016). Text mining is a process used to extract useful information from unstructured text, such as features, patterns and relationships of interest (Ignatow and Mihalcea, 2017). With the advancements in natural language processing (NLP) and neural network technology, the technique of text mining is evolving. Recently, highly advanced neural language models (NLM), such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT), have been developed. These models, which combine neural network technology and big data, have demonstrated outstanding performance in text mining (Devlin et al., 2018; Floridi and Chiriatti, 2020).
Despite changes in the corporate management, investment and technical environment, few textual analysis studies simultaneously considered the ESG and SDG-related contexts described in Form 10-K. This study aims to provide readers (e.g. consumers and investors) with derived information that can be used in ESG performance evaluation by analyzing Form 10-K to quantify and visually describe the relationship between ESG and SDGs implied in the text. In addition, this study also aims to present initial evidence for the automation potential of processes that identify which companies are in alignment with the SDGs. To this end, this study proposed a novel quantitative method that considered ESG keywords (Baier et al., 2020) included in the sentence and the context of the sentence using BERT-based NLM. Existing related studies that employed textual analytics mainly used lexicon and keywords-based methods for measuring frequency, which have the problem of counting words used in unrelated sentences even when keywords are well-defined for a particular domain (Loughran and McDonald, 2015). The method proposed in this study could be a solution to this issue by extracting and refining only meaningful sentences from Form 10-K, composed of large amounts of text.
The detailed tasks of this study are as follows. First, the ESG-related sentences extracted from Form 10-K were compared with 17 SDGs statements to be quantified. As a comparison method, similarity analysis based on sentence embeddings using Sentence-Transformers (Reimers and Gurevych, 2019) was performed. Second, the classifier was developed that probabilistically categorizes sentences into “SDG-related ESG” by considering the relationship between ESG and SDGs statements. To achieve this, a training dataset was constructed using sentence similarity data and FinBERT (Araci, 2019), which was trained with more than 29 million words extracted from 46,143 financial documents, was fine-tuned. Third, the changes and trends in ESG management activities over the past 10 years reflected on Form 10-K were investigated by diversifying them by SDG-related ESG content, company size, industry and individual company. The following assumptions were made based on the characteristics of disclosure for the interpretation of the analysis results. The more sentences conform to SDGs statements on a single Form 10-K, that is, “SDG-related ESG sentences,” the higher the visibility of ESG management activities. Quantitative changes in SDG-related ESG sentences are related to the circumstances of the times (e.g. the COVID-19 pandemic and investors’ increasing demand for sustainable investment).
In this sense, the research questions (RQs) are as follows:
This study is organized as follows. In Section 2, the relationships between ESG, SDGs and Form 10-K are reviewed, and previous studies that have conducted text analysis of data published by companies, such as Form 10-K and CSR reports, are investigated. Section 3 describes the data collection process, data preprocessing procedure, similarity analysis, fine-tuning FinBERT and SDG-related ESG classification. Section 4 presents analysis results in figures and charts. Finally, in Section 5, the conclusion, implications and limitations of this study are discussed.
2. Literature review
2.1 Relationship between ESG, SDGs and corporate disclosures
The demand for socially responsible investment (SRI) and sustainable investment, which selects investment targets based on sustainability and ESG factors, has been rising. This trend is reflected in the growth of related markets (Saad and Strauss, 2020) and increased interest from international organizations and regulatory authorities. In response to this growing demand for ESG-compliant investments, various efforts have been made to improve transparency and protect investors through implementing ESG-related legislation, regulatory mechanisms and evaluation standards (Bowley and Hill, 2022). One notable example of this trend is the decision by the European Union to implement mandatory ESG disclosure for companies of a certain size or larger from 2024 (European Parliament, 2022). Similarly, SEC proposed a rule in March 2022 to include ESG-related factors and strategies in regular corporate disclosures, such as Form 10-K (SEC, 2022).
The UN SDGs also play an important role in this context, as they contain 17 explicit goals and 169 targets for solving social and environmental problems. The concepts underlying the SDGs are closely aligned with those of ESG, and the two frameworks are mutually reinforcing (Berenberg, 2018; Delgado-Ceballos et al., 2022). Therefore, companies that perform well on ESG factors are more likely to make progress towards the SDGs. In the absence of international standards for measuring ESG performance, the SDGs have been used as general guidelines for corporate disclosure and evaluation standards in terms of sustainability (Blasco et al., 2018). Many multinational corporations, such as Microsoft and Samsung Electronics, have recognized the importance of integrating the SDGs into their ESG management strategies to improve their non-financial performance (Microsoft, 2022; Samsung Electronics, 2022).
Table 1 shows the mapping (Berenberg, 2018; Markopoulos and Barbara Ramonda, 2022) between 17 UN SDGs’ statements and ESG. The number of targets varies across SDGs. For example, Goal 9, “Industry, Innovation and Infrastructure,” has eight targets to attain the goal and the first target is “Implement the 10-year framework of programmes on sustainable consumption and production, all countries taking action, with developed countries taking the lead, taking into account the development and capabilities of developing countries.” Due to the comprehensive nature of the three elements that make up ESG, they are linked in an n:m relationship with SDGs statements. Therefore, from a quantitative measurement perspective, the overlapping mapping of SDGs onto each factor of ESG is considered.
Corporate disclosures, such as Form 10-K, are considered to be of considerable value as research materials and are believed to aid in the decision-making process of investors (Bodoff and Zhang, 2003). Chi and Shanthikumar (2018) argue that the trading activity of retail investors is more closely related to the search volume for Form 10-K and 10-Q (quarterly report) than the search volume for Google. Form 10-K, an annual report, includes both audited financial statements and a significant amount of qualitative and quantitative corporate information. Due to these features, Form 10-K has been actively utilized in various studies that approach it with textual analysis methods, such as identifying risk factors (Wang et al., 2019), identifying distressed companies (Gandhi et al., 2019), predicting performance (Azimi and Agrawal, 2021) and machine trade (Schmitz et al., 2023). As interest in ESG and SDGs, the precursor benchmarking elements of ESG, increases in terms of social and economic aspects, corporate disclosures such as Form 10-K will be a useful data source to identify and track corporate ESG management activities. Additionally, as corporate disclosures regarding ESG performance are increasing (Alareeni and Hamdan, 2020), it is expected that research analyzing the qualitative aspects of ESG in Form 10-K will also increase. These points will be the reason for this study’s significance, necessity and value. The following section summarizes the content and results of previous studies that applied text analysis methods to ESG subjects.
2.2 Textual analysis studies for ESG subjects
As interest in ESG issues continues to grow, the academic community has conducted various research aimed at measuring, evaluating and comparing the performance of organizations with respect to ESG. One of the major endeavors is to identify efficient and effective methods for extracting ESG-related information from corporate disclosures, CSR reports, news articles and other forms of text data. Given the prevalence of big data in contemporary society, using textual analysis techniques to study ESG is an up-and-coming area of research. However, despite the potential of this approach, the number of studies utilizing text analysis methods to examine ESG remains relatively limited. The following summarizes existing literature using textual analysis techniques to investigate ESG.
Baier et al. (2020) attempted to quantify unstructured narrative content related to ESG from the perspective of SRI. To accomplish this, the authors analyzed the texts in Form 10-K and proxy statements of the 25 largest companies in the S&P 100 index from multiple perspectives and dimensions. The study proposed 482 ESG words, divided into 10 categories and 40 subcategories, and revealed the ESG words accounted for a proportion of approximately four percentages in the analyzed documents. The study also found that governance is overwhelmingly more prevalent among the three elements of ESG than environmental and social topics.
Borms et al. (2021) aimed to support the decision-making of SRI investors by detecting text-based ESG signals from news articles. They selected 49 ESG seed words based on over 4 million Dutch-language news texts written between 1996 and 2019. The authors then performed word embeddings on the news corpus using Text2Vec and calculated cosine similarity between the embeddings of the ESG seed words. This resulted in identifying 539 ESG-related words, which were subsequently utilized for ESG signal detection. To verify the proposed methodology, the authors compared the detection results with ESG scores provided by an external data provider, and found the methodology to be valid and reliable.
Jiang et al. (2022) proposed a hybrid methodology that combines text mining and manual work to extract simplified ESG taxonomies. The authors extracted text from the 2010 to 2020 ESG reports of European oil and gas companies, and then performed lemmatizing and tokenizing on the extracted text. Based on the structure of Global Reporting Initiative (GRI) standards, they developed ESG word lists from the extraction results. By comparing the frequency of ESG words and GRI topics through manual verification, the authors claimed that the proposed method is valid, with over 90% agreement.
Serafeim and Yoon (2022) conducted a study to investigate the effect of positive or negative ESG news on stock prices by analyzing 3,126 companies’ cases. The study revealed that investors responded selectively to the news with a high likelihood of impacting the company’s fundamentals, and to issues that were financially important. The study also found that the magnitude of the response increased as the news content was positively related to social capital issues.
Mehra et al. (2022) utilized NLM, specifically BERT, to extract ESG-related information from easily accessible data sources such as corporate disclosures, reports and news articles. The study used the Knowledge Hub of the Accounting for Sustainability project’s ESG corpus and Form 10-Q filings from S&P 500 companies as its training data. BERT was fine-tuned to perform tasks determining task 1 whether a change in ESG-related text was present or not and task 2 whether this change was positive or negative. The research achieved a test accuracy of 0.67 for task 1 and 0.79 for task 2. Other studies have also utilized BERT for tasks such as stock volatility prediction (Guo et al., 2020), classifying the relevance of a text sentence to ESG (Raman et al., 2020), and developing a method to extract textual evidence of ESG by automatically labeling sentences (Kannan and Seki, 2023).
Based on the literature review of previous research, it can be inferred that one of the common objectives of extracting ESG-related information is to support investors’ decision-making. In terms of analytical methods, in addition to lexicons and keywords-based approaches, there has been an increasing trend of utilizing NLM, such as BERT, in recent textual analysis studies. However, research that systematically tracks and quantifies ESG-related disclosures over a long-term timeframe through the large-scale data analysis is still limited. Furthermore, studies that provide a detailed analysis of the characteristics of ESG-related disclosures across various industries and without any size limitations of the companies are currently scarce. Therefore, this study is differentiated from previous research, both theoretically and practically.
3. Methods
3.1 Research framework
Figure 1 illustrates the data analysis process of this study. The first step of the process involves the data collection of Form 10-K from the SEC’s Electronic Data Gathering, Analysis, and Retrieval System (EDGAR). The second step consists of a data preprocessing phase, which includes removing HTML tags, adjusting the data time frame, extracting text related to ESG and identifying individual sentences. In the third step, a sentence similarity analysis is conducted by comparing the embeddings of sentences related to ESG issues with those of the SDGs statements. The fourth step involves the classification of ESG-related sentences utilizing Fine-tuned FinBERT. Finally, in the fifth step, the relationship between ESG contexts and SDGs statements is quantitatively examined based on the results of the classification process.
3.2 Data preprocessing
3.2.1 Collection of form 10-Ks
The SEC provides online access to corporate disclosure data through the EDGAR system, as well as a variety of search options. This study utilized the submission bulk, which comprises the disclosure history of individual companies. The temporal scope of the study was from January 2010 to September 2022, and a list of companies that disclosed Form 10-K within this period was extracted from the submission bulk. The central index key (CIK) is a unique number that distinguishes companies, and the submission bulk is a compressed version of the CIK files. The CIK files follow the JSON (JavaScript Object Notation) structure and serve as metadata for the disclosure history, as well as containing additional information such as company names and business addresses. A total of 16,741 companies were found to have disclosed Form 10-K during the designated period, and the number of collectable Form 10-K was 95,943.
3.2.2 The adjustment of the data time frame
It is crucial to ensure temporal consistency to analyze data in a time-series manner. However, states in the United States have varying accounting year-end dates and filing deadlines for public disclosures based on the corporation’s size. They have potential extensions due to special circumstances, which may reflect a discrepancy between the timeliness of the information contained in Form 10-K and submission dates. Fortunately, the CIK file includes the report date or period of report information, which can be used to standardize the time frame of the data and ensure temporal consistency. The following is a description of the procedure and method employed.
Step 1) Calculate the quarter to which the Form 10-K’s report date belongs.
Step 2) If the quarter calculated in step 1 is one of the first three quarters, add 1 to the quarter. If it is the fourth quarter, keep the quarter as 1.
Step 3) If the quarter calculated in step 2 is one of the second, third, or fourth quarters, use the year of Form 10-K’s report date as is, and if it is the first quarter, add 1 to the year.
3.2.3 Extraction of candidate sentences
Form 10-K is typically in HTML format, and various unnecessary data like financial statements were included with HTML tags, which is unnecessary for this study. Therefore, the first step was to utilize the Beautiful Soup library to remove all unnecessary data, such as HTML tags, headings, financial statements, and so on. Subsequently, the Natural Language Toolkit (NLTK) library’s Sentence Tokenizer was used to split the text of each report into sentence units. Among these individual sentences obtained through the separation process, the sentences containing 482 keywords of the ESG dictionary (Baier et al., 2020) were parsed. In this process, the sentences were extracted sequentially, enabling the use of sentence order information during data analysis.
Finally, morphological analysis applying the spaCy library was carried out to identify if the extracted sentences were proper English. Since correct English sentences are in the form of a clause composed of at least one noun and a verb, the texts satisfying this condition were extracted. Despite undergoing the aforementioned process, some texts were not in the form of correct English sentences. Through multiple reviews, it was observed that errors frequently occurred in sentences consisting of four or fewer words. Thus, all sentences with four or fewer words were removed. The number of remaining sentences was 27,661,489, which were defined as candidate sentences and were utilized for the sentence similarity analysis described in the next section.
3.3 Sentence similarity analysis
To compare the similarities between sentences, the texts were converted into numerical values. Word embedding is a method that converts sentences into vectors that best represent the words composing each sentence. This study extracted sentence embedding information using the Sentence Transformers (Reimers and Gurevych, 2019). Sentence Transformers is a model that has been improved to provide better sentence representations by using the output vectors from BERT.
The similarity between sentences was calculated based on Cosine Similarity, a commonly used method in text comparison (Bengfort et al., 2018). When two sentence vectors are completely identical, Cosine Similarity has a value of 1 and a value of 0 in the opposite case. The closer the value is to 1, the more similar the sentences are. Comparing a candidate sentence to SDGs statements, the SDG with the highest similarity value was considered to be the prevailing SDG for the sentence. For example, the prevailing SDG was Goal 13 with a similarity value of 0.8 (see Table 2) when calculating the similarity between the sentence “We are committed to protecting our environment and addressing climate change issues through product responsibility, water stewardship, and GHG emissions reduction” from the 2022 Form 10-K of fashion company GUESS Inc. and 17 SDGs statements. The following describes the sentence similarity analysis process, and Table 3 shows the distribution of prevailing SDGs for candidate sentences.
Step 1) Calculate sentence embeddings for 17 SDGs statements.
Step 2) Calculate sentence embeddings for candidate sentences.
Step 3) Calculate the similarity between the sentence embeddings of Steps 1 and 2.
Step 4) Identify the SDG with the highest similarity for each candidate sentence. If the similarity value is 0.5 or higher, classify it as the corresponding SDG (i.e. prevailing SDG) for the candidate sentence. If the value is less than 0.5, classify it as “No relation.”
3.4 Classification of ESG texts
3.4.1 Summary of the fine-tuning process
BERT (Bidirectional Encoder Representations from Transformers) is a language model that was introduced by Google in 2018, utilizing the Transformer Attention Mechanism. As a pre-trained language model (PLM), BERT can easily be fine-tuned to perform specific tasks, leading to the creation of various PLMs based on BERT (Arslan et al., 2021). FinBERT (Araci, 2019) is a PLM that was trained on financial documents, incorporating over 29 million words that appeared in the financial domain. This model achieved higher classification accuracy in financial sentiment classification compared to traditional BERT.
The objective of fine-tuning in this study is to create a classifier based on FinBERT to classify ESG-linked SDGs. Considering the n:m relationship between SDG and the factors of ESG, discrete classes were defined to ensure unambiguous classification, referring to the mapping in Table 1. The defined classes include E, S, G, ES, EG, SG, ESG and None. For example, Goal 2, “Good health and well-being,” is linked to the social factor in ESG. If the created classifier functions properly, when given a similar context to “End hunger, achieve food security and improved nutrition and promote sustainable agriculture,” which is a statement of Goal 2, the classifier should classify the context as class S. For statements not related to ESG, the classifier should classify them as None. The next part explains the construction of the training dataset, the setup for training, the evaluation of the model’s performance, and the SDG-related ESG classification process.
3.4.2 Building the training dataset
In this study, fine-tuning was performed as a supervised learning method, thus requiring a labeled training dataset. The construction of the training dataset was founded on the premise that sentences with higher similarity to each SDGs statement would exhibit greater congruence in terms of context and meaning. The construction method involved the following steps; firstly, candidate sentences with a similarity of at least 0.8 to SDGs statements were randomly extracted from the sentence similarity data, and these sentences served as the foundational elements of the training dataset. Subsequently, 169 SDGs target sentences were added to the training dataset. The 169 SDGs target sentences describe specific ways to achieve each SDG, thus providing high-quality word embeddings that enhance classification performance. To classify sentences unrelated to ESG, 1,000 randomly selected sentences with a low similarity (below 0.5) to all 17 SDGs statements were extracted from among sentences with a prevailing SDG of “No relation.” These sentences were labeled as class None and added to the training dataset. Table 4 displays the number of sentences in the training dataset for each SDG-related ESG class.
The training dataset was divided into the train set and the test set in a ratio of 8:2. To alleviate the imbalanced number of sentences between classes, random oversampling was performed based on class SG, which had the largest sample size (n = 609). The train set was further divided into the train set and the validation set in a ratio of 9:1. This completed the preparation of the data for training.
3.4.3 Training and evaluations
The objective of fine-tuning in this study was to modify the original output layer of FinBERT for sentiment classification (i.e. positive, negative, neutral) into classifying eight SDG-related ESG classes. The main hyper-parameters for training were set up: learning rate = 2e−5, batch size = 128, epochs = 10, optimizer = AdamW, and loss function = cross-entropy. Evaluation accuracy was used as an indicator for the performance evaluation of training. Figure 2 shows the training results. Among 10 epochs, (1) the evaluation loss was at its lowest (0.11) in epoch 9, and (2) the evaluation accuracy was at its highest (0.98) in the same epoch. Therefore, the model created from the training results up to epoch 9 was selected as the final classifier for this study. The results of evaluating the test dataset using the model showed an overall accuracy of 0.89. Most sentences unrelated to ESG were classified as class None (see Table 5).
Additionally, the classification performance on the SDGs statements was tested, and Table 6 illustrates the results. While the accuracy of 15 out of the 17 SDGs was 88.2%, the Goal 5 and 10 exhibited discrepancies when comparing the results of the classifier developed in this study with the ESG-SDG mapping.
3.4.4 The classification of SDG-related ESG sentence
The classification targets were 8,326,968 sentences with a similarity value of at least 0.5 with one of the 17 SDGs statements, which were previously completed through similarity analysis (see Table 3). The classification task was performed by applying a classifier with fine-tuned FinBERT on the candidate sentences. This study considered sentences properly classified if the prediction probability of their class was 0.8 or higher and defined them as SDG-related ESG sentences. Sentences with a classification result of None were considered unrelated to ESG and excluded from the analysis, regardless of their prediction probability. The result of the classification task indicated that there were 1,236,708 SDG-related ESG sentences. Table 7 compares the number of candidate sentences and SDG-related ESG sentences.
From the classified SDG-related ESG sentences, 100 samples for each of the 7 classes, totaling 700 samples, were randomly selected and manually verified by the researchers. Table 8 represents the results of manual verification, with an overall precision of 81% for the 700 samples. This figure serves as a reference point for accuracy in interpreting the data analysis results reported in the following section.
4. Results and findings
This section discusses the quantitative changes in SDG-related ESG sentences in three ways with respect to the three RQs in the study: (1) overall changes in an industry, (2) differences between industries and (3) changes for individual companies.
4.1 Distribution of SDG-related ESG sentences
Table 9 displays the annual distribution of SDG-related ESG sentences by class. A trend of increasing sentences in all classes each year is observed.
Figure 3 illustrates the yearly variation of the average of sentences per class in Form 10-K filings. A general upward trend is observed, with a more pronounced slope in 2020. Figure 4 presents the results of the min-max normalization applied to each SDG-related ESG class. The upward trend is maintained and the slope of all classes in 2020 appears to be even more accentuated than in Figure 3 due to the result of the normalization.
The data presented above demonstrate the distribution and time series trend of SDG-related ESG sentences over the past decade, which is the RQ1 of the study. The three notable features that stand out from the graph are as follows.
First, the number of sentences corresponding to class ESG is greater than other classes. Class ESG encompasses Goal 9, “Industry, Innovation and Infrastructure,” and Goal 12, “Responsible Consumption and Production,” and these goals have a close relationship with corporate management activities. Therefore, it is reasonable to conclude that the dominance of class ESG is a natural outcome as this study is based on Form 10-K. Second, the slope of the upward curve for all classes increased further in the wake of the COVID-19 pandemic in 2020. The pandemic heightened the investment community’s concern over the financial system and increased the demand for transparency and consistency of ESG-related information (Adams and Abhayawansa, 2022). Companies worldwide responded to this demand by aligning their ESG management, practices and disclosures with investor needs (Atkins et al., 2022; Engelhardt et al., 2021). As a result, ESG-related information would have been recorded in disclosures such as Form 10-K. Third, the slope of classes, including social factors, is increasing, particularly class SG, which showed a two-fold increase in 2021 compared to 2020. The COVID-19 pandemic reinforced the demand for CSR, and many companies competed to respond to the public’s expectations for resolving urgent global social issues (Carroll, 2021). The trend in the aforementioned graph can be interpreted as indirectly depicting the historical context.
4.2 Comparisons across industries
This section examines the distribution and time series trends of SDG-related ESG sentences by industry and company size in relation RQ2. In the study, the classification of the industry was based on the division structure of the U.S. Department of Labor. Companies were categorized into one of nine industries based on their standard industrial classification (SIC) code. Figures 5 and 6 visualized the distribution of SDG-related ESG sentences for each industry, quantified into two categories, quantity and normalized. Figure 5 shows that the largest number of SDG-related ESG sentences was in the Manufacturing sector, which comprises a large number of companies, while the lowest was in the Agriculture, Forestry and Fishing sector. Figure 6 shows the trend of normalized SDG-related ESG sentences, which generally shows an upward pattern, although differences depend on the industry.
Figure 7 depicts the proportion of SDG-related ESG sentences across different sectors, categorized by class. The graph patterns of classes S, G, SG and ESG exhibit similarities, while classes E, ES and EG present a characteristic feature with a significant representation of the TCE and the Mining industries. The SDGs statements belonging to classes E, ES and EG encompass messages concerning environmental factors such as climate change, ecosystem, energy, forests and water. Enterprises that produce energy and metals fall under the category of environmentally sensitive industries and are subject to stringent regulations and laws aimed at mitigating environmental concerns (Naeem et al., 2022). Therefore, there is a high degree of pressure on the requirement of disclosure in relation to relevant management activities. Research on ESG disclosure transparency for S&P 500 companies indicates that the environmental factor scores of the TCE sector are higher than other industries (Tamimi and Sebastianelli, 2017). Based on this information, it can be deduced that the distinctive pattern observed in classes E, ES and EG is due to these factors.
Figure 8 compare the distribution and time-series trends in the average SDG-related ESG sentences per year for the top 5% companies (i.e. large-cap) and the bottom 5% companies (i.e. small-cap) in terms of market capitalization at the time of this study. Figure 9 reveals a gentle upward trend in the yearly average SDG-related ESG sentences for both large-cap and small-cap companies, with the slope becoming steeper for large-cap companies starting in 2020. Table 10 shows the Pearson correlation between SDG-related ESG sentences in the 2022 Form 10-K filings and market capitalization. The overall correlation was 0.42 (p < 0.001), and the financial sector showed the highest correlation of 0.56 (p < 0.001). The results of this study can be seen as data-based empirical evidence and are consistent with the notion that larger companies are more sensitive to consumer and public opinion (Fombrun and Shanley, 1990) and have greater financial and public disclosure capabilities to engage in CSR (Ting, 2021) and ESG (Gjergji et al., 2021) activities.
4.3 Changes in corporate ESG disclosures
Form 10-K serves as an effective means of communicating information to create a favorable corporate image. The contents of Form 10-K also change in response to negative events, social demands and market conditions (Dyer et al., 2017; Erickson et al., 2011). The followings are examples of the qualitative and quantitative capture of the changes in the contents of Form 10-K through data analysis in this study.
First, the example is PVH Corp., which belongs to the apparel industry that is based on global supply chains and is labor intensive. Through the data analysis method of this study, the newly added sentence in PVH Corp.’s 2013 Form 10-K, “We actively work to educate our associates and partners and improve factory conditions, as well as continue to invest in the communities where we do business” was identified and the class of the sentence was SG. In April 2013, a tragedy occurred in Bangladesh when the Rana Plaza clothing factory collapsed, killing more than a thousand people. The unethical labor issues in the related industry were greatly highlighted (Barua and Ansary, 2017), and as a result, there was a trend of increased corporate disclosure regarding social issues such as workplace safety (Akbar and Deegan, 2021). At the time, PVH Corp. was the first company to sign the international agreement, Accord for Fire and Building Safety in Bangladesh, in July 2013 to prevent such incidents from happening again. It can be interpreted that this information was disclosed to respond to the investment sentiment that was affected by the negative event in the industry and to improve the corporate image.
The next is an example of a company in the aviation industry that experienced the greatest management crisis due to the COVID-19 pandemic. This study extracted sentences sequentially from Form 10-K, resulting in the inherent sequence appearance of the SDG-related ESG sentences. By utilizing the sequences and classes of the sentences, the distribution of the sentences can be mapped visually to trace the time series changes and characteristics of the companies. Figure 9 shows the sentence maps from 2018, before the COVID-19 pandemic, to the present. The x-axis represents the order of appearance of sentences in Form 10-K of the respective year, and the number on the right side of the axis indicates the total number of SDG-related ESG sentences. There was little quantitative change in 2018 and 2019. However, between 2020 and 2022, the number of SDG-related ESG sentences increased annually. As previously stated, it is likely a result of the companies’ continuous expansion of ESG-related disclosures to decrease investment risk and increase investment appeal for investors amid the global crisis caused by the COVID-19 pandemic (Adams and Abhayawansa, 2022).
The above discussion addresses the underlying causes responsible for the changes in SDG-related ESG disclosures in response to two major events about RQ3. Investigating changes in corporate disclosures helps to understand companies and is also necessary for investment decision-making. The data analysis and visualization results presented provide a valuable means to explore SDG-related ESG topics.
5. Conclusion
Based on the conceptual connection between ESG, non-financial elements of a company and the UN SDGs statements for resolving global issues, this study extracted ESG-related sentences and quantified them from company disclosures, considered the most reliable corporate data (Mehra et al., 2022). The study also explored how the context of ESG changed over the past decade in corporate disclosures and analyzed the changes and characteristics from a multi-dimensional perspective. A novel method considering the context was presented for extracting and classifying valuable sentences as data by applying Sentence Transformers and FinBERT. 93,277 Form 10-K disclosures from January 2010 to September 2022 were collected, and sentence-embedding information of candidate sentences containing ESG keywords was obtained. A classifier that classifies eight ESG classes with an overall accuracy of 89% was created by fine-tuning FinBERT. Utilizing the classifier, the ESG classes of candidate sentences with high similarity to the SDGs statements were classified, and these sentences were defined as SDG-related ESG sentences.
The analysis results quantitatively and visually confirm the increase in corporate disclosures on ESG performance (Alareeni and Hamdan, 2020). The trend of SDG-related ESG sentences over the past decade was a steadily increasing upward curve, although it was not dramatic. The number of sentences increased by more than two times from 2010 to 2022. Large-cap companies had relatively more SDG-related ESG sentences than small-cap companies. In a comparison by industry, the industry with the highest number of SDG-related ESG sentences appearing in a Form 10-K for 2022 was the TCE sector, with an average of 42.11 sentences. In contrast, the FIR sector had the least number of sentences with an average of 17.38. In the analysis of changes in the content of the company’s disclosures, the differences were examined by comparing the Form 10-Ks disclosed before and after two historical events (i.e. the Rana Plaza collapse and the COVID-19 pandemic) and discussing plausible reasons for the differences.
The present study sheds light on various implications for research and practice. Firstly, as one of the few studies that approached ESG-related research using a machine learning-based text analysis method, it revealed the relationship between SDG and ESG, which is not explicitly stated in corporate disclosures. This finding can provide valuable information and insights, as well as a foundation for further research, on the changes, characteristics and directions of individual companies and the broader industry (Ignatow and Mihalcea, 2017). Secondly, the quantified results of ESG disclosures can be utilized as reference materials in decision-making for SRI and sustainable investment. In the context of uncorrelated grades or scores of ESG performance provided by ESG data providers such as MSCI and Refinitiv (Boffo and Patalano, 2020), this study provides consistency in the data. Additionally, the text analysis process proposed in the study can serve as a basis for providing investors with time-sensitive information as it is automated. Finally, in a social context where competition among companies is intensifying, and the importance of ESG management and implementation is emphasized, the results and discussions of the study suggest the need for long-term ESG disclosure plans for the sustainable management of companies.
This study has several limitations. Form 10-K is written by the companies and its reliability and the amount vary depending on the company, which may result in bias the analysis results. Additionally, it should be noted that the study only mechanically measures and evaluates similarities with SDGs statements and analyzes those with high similarities. Therefore, adding a qualitative analysis of whether a company actually contributes to SDGs and ESG performance is recommended. If the remaining contents of Form 10-K, which are not addressed in this study, are quantified from an ESG perspective and added to the results of this study, more comprehensive information may be obtained.
Figure 1
Research framework for the data analytics processing
[Figure omitted. See PDF]
Figure 2
The training results of fine-tuned the FinBERT
[Figure omitted. See PDF]
Figure 3
Time-series of the average SDG-related ESG sentences in Form 10-Ks
[Figure omitted. See PDF]
Figure 4
Time-series of the average SDG-related ESG sentences (Normalized) in Form 10-Ks
[Figure omitted. See PDF]
Figure 5
Time-series of SDG-related ESG sentences (Quantity) in Form 10-Ks by industry
[Figure omitted. See PDF]
Figure 6
Time-series of SDG-related ESG sentences (Normalized) in Form 10-Ks by industry
[Figure omitted. See PDF]
Figure 7
Time-series of the SDG-related ESG sentences by class and industry
[Figure omitted. See PDF]
Figure 8
Time-series of the yearly average SDG-related ESG sentences of large-cap and small-cap
[Figure omitted. See PDF]
Figure 9
An example of mapping SDG-related ESG sentences from 2018 to 2022
[Figure omitted. See PDF]
Table 1
Statements of the 17 SDGs and mapping with ESG factors
| Goals | SDGs statements | The number of targets | ESG factors |
|---|---|---|---|
| Goal 1: No poverty | End poverty in all its forms everywhere | 7 | S |
| Goal 2: Zero hunger | End hunger, achieve food security and improved nutrition and promote sustainable agriculture | 8 | S |
| Goal 3: Good health and well-being | Ensure healthy lives and promote well-being for all at all ages | 13 | S |
| Goal 4: Quality education | Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all | 10 | S |
| Goal 5: Gender equity | Achieve gender equality and empower all women and girls | 9 | S, G |
| Goal 6: Clean water and sanitation | Ensure availability and sustainable management of water and sanitation for all | 8 | E, S |
| Goal 7: Affordable and clean energy | Ensure access to affordable, reliable, sustainable and modern energy for all | 5 | E |
| Goal 8: Decent work and economic growth | Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all | 12 | S, G |
| Goal 9: Industry, innovation and infrastructure | Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation | 8 | E, S, G |
| Goal 10: Reduced inequalities | Reduce inequality within and among countries | 10 | S |
| Goal 11: Sustainable cities and communities | Make cities and human settlements inclusive, safe, resilient and sustainable | 10 | E, G |
| Goal 12: Responsible consumption and production | Ensure sustainable consumption and production patterns | 11 | E, S, G |
| Goal 13: Climate Action | Take urgent action to combat climate change and its impacts | 5 | E, G |
| Goal 14: Life below water | Conserve and sustainably use the oceans, seas and marine resources for sustainable development | 10 | E |
| Goal 15: Life on land | Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification and halt and reverse land degradation and halt biodiversity loss | 12 | E |
| Goal 16: Peace, justice and strong institutions | Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels | 12 | S, G |
| Goal 17: Partnership for the goals | Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development | 19 | G |
Note(s): Edited from the study of Berenberg (2018), Markopoulos and Barbara Ramonda (2022), and contents of United Nations’ official websites, https://sdgs.un.org/goals. E = Environmental; S = Social; G = Governance
Source(s): Table by authors
Table 2
Evaluation of sentence similarity with cosine similarity
| SDGs | Cosine similarity | SDGs | Cosine similarity | SDGs | Cosine similarity |
|---|---|---|---|---|---|
| Goal 1 | 0.34 | Goal 7 | 0.48 | Goal 13 | 0.80 |
| Goal 2 | 0.66 | Goal 8 | 0.39 | Goal 14 | 0.69 |
| Goal 3 | 0.43 | Goal 9 | 0.60 | Goal 15 | 0.74 |
| Goal 4 | 0.45 | Goal 10 | 0.43 | Goal 16 | 0.53 |
| Goal 5 | 0.34 | Goal 11 | 0.52 | Goal 17 | 0.68 |
| Goal 6 | 0.61 | Goal 12 | 0.60 |
Note(s): Cosine similarities were calculated between 17 SDGs statements and the sample sentence, “We are committed to protecting our environment and addressing climate change issues through product responsibility, water stewardship, and GHG emissions reduction”
Source(s): Table by authors
Table 3
Distribution of prevailing SDGs in candidate sentences
| SDGs goal number | The number of sentences | SDGs | The number of sentences | SDGs | The number of sentences |
|---|---|---|---|---|---|
| Goal 1 | 148,560 | Goal 7 | 97,790 | Goal 13 | 2,187,925 |
| Goal 2 | 191,750 | Goal 8 | 277,627 | Goal 14 | 23,784 |
| Goal 3 | 6,415 | Goal 9 | 1,594,577 | Goal 15 | 258,770 |
| Goal 4 | 57,713 | Goal 10 | 1,506,957 | Goal 16 | 235,146 |
| Goal 5 | 14,338 | Goal 11 | 39,529 | Goal 17 | 383,884 |
| Goal 6 | 267,115 | Goal 12 | 1,035,088 | No relation | 19,334,521 |
Note(s): “No relation” indicates that all values of cosine similarities were less than 0.5 by SDG
Source(s): Table by authors
Table 4
The training dataset of each SDG-related ESG class
| Classes | The number of sentences | Mapped UN SDG goals |
|---|---|---|
| Class E | 178 | Goal 7, Goal 14, Goal 15 |
| Class S | 525 | Goal 1, Goal 2, Goal 3, Goal 4, Goal 10 |
| Class G | 271 | Goal 17 |
| Class ES | 87 | Goal 6 |
| Class EG | 282 | Goal 11, Goal 13 |
| Class SG | 609 | Goal 5, Goal 8, Goal 16 |
| Class ESG | 475 | Goal 9, Goal 12 |
| Class None | 1,000 | Not available |
| Total number of classes | 3,427 |
Note(s): E = Environmental; S = Social; G = Governance
Source(s): Table by authors
Table 5
Classification results of the test dataset
| SDG-related ESG classes | ||||||||
|---|---|---|---|---|---|---|---|---|
| E | S | G | ES | EG | SG | ESG | None | |
| Precision | 0.88 | 0.90 | 0.76 | 0.80 | 0.78 | 0.90 | 0.84 | 0.99 |
| Recall | 0.85 | 0.95 | 0.89 | 0.95 | 0.92 | 0.82 | 0.71 | 0.98 |
| F1 score | 0.86 | 0.92 | 0.82 | 0.87 | 0.84 | 0.86 | 0.77 | 0.98 |
| Overall accuracy | 0.89 | |||||||
Note(s): None indicates the class which has no relationship with SDG-related ESG contexts
Source(s): Table by authors
Table 6
The result of classification for 17 SDGs statements
| SDGs | Actual class | Predicted class | SDGs | Actual class | Predicted class | SDGs | Actual class | Predicted class |
|---|---|---|---|---|---|---|---|---|
| Goal 1 | S | S (True) | Goal 7 | E | E (True) | Goal 13 | EG | EG (True) |
| Goal 2 | S | S (True) | Goal 8 | SG | SG (True) | Goal 14 | E | E (True) |
| Goal 3 | S | S (True) | Goal 9 | ESG | ESG (True) | Goal 15 | E | E (True) |
| Goal 4 | S | S (True) | Goal 10 | S | SG (False) | Goal 16 | SG | SG (True) |
| Goal 5 | SG | S (False) | Goal 11 | EG | EG (True) | Goal 17 | G | G (True) |
| Goal 6 | ES | ES (True) | Goal 12 | ESG | ESG (True) |
Source(s): Table by authors
Table 7
Classification results for candidate sentences
| SDG-related ESG classes | ||||||||
|---|---|---|---|---|---|---|---|---|
| Sentences | E | S | G | ES | EG | SG | ESG | None |
| Candidate | 182,638 | 239,045 | 240,679 | 256,642 | 2,798,214 | 451,877 | 1,335,377 | 2,822,496 |
| SDG-related ESG | 36,464 | 106,371 | 87,875 | 67,475 | 156,200 | 225,155 | 557,168 | |
Source(s): Table by authors
Table 8
Verifying samples of SDG-related ESG sentences
| SDG-related ESG classes | ||||||||
|---|---|---|---|---|---|---|---|---|
| E | S | G | ES | EG | SG | ESG | Overall | |
| Precision | 0.85 | 0.85 | 0.75 | 0.90 | 0.80 | 0.79 | 0.73 | 0.81 |
Note(s): One hundred samples were selected randomly from each SDG-related ESG class
Source(s): Table by authors
Table 9
The annual distribution of SDG-related ESG sentences in Form 10-Ks by class
| Year | The number. of form 10-Ks | SDG-related ESG classes | ||||||
|---|---|---|---|---|---|---|---|---|
| E | S | G | ES | EG | SG | ESG | ||
| 2010 | 7,985 | 2,523 | 5,796 | 3,972 | 3,353 | 9,059 | 11,971 | 33,687 |
| 2011 | 7,906 | 2,708 | 6,284 | 4,399 | 4,019 | 9,916 | 13,005 | 35,613 |
| 2012 | 7,721 | 2,625 | 6,292 | 4,415 | 4,300 | 9,810 | 12,557 | 34,458 |
| 2013 | 7,558 | 2,886 | 6,186 | 4,844 | 4,903 | 10,690 | 13,612 | 37,244 |
| 2014 | 7,621 | 2,676 | 6,852 | 5,170 | 5,389 | 11,247 | 14,602 | 39,047 |
| 2015 | 7,719 | 2,768 | 7,527 | 5,744 | 5,612 | 11,458 | 15,372 | 41,682 |
| 2016 | 7,445 | 2,719 | 7,308 | 5,808 | 5,473 | 11,181 | 14,711 | 41,087 |
| 2017 | 7,149 | 2,591 | 7,175 | 5,860 | 5,439 | 10,779 | 14,616 | 41,676 |
| 2018 | 6,959 | 2,691 | 7,130 | 6,162 | 5,236 | 10,924 | 14,635 | 42,066 |
| 2019 | 6,860 | 2,619 | 7,144 | 6,620 | 5,124 | 10,671 | 14,743 | 42,352 |
| 2020 | 6,683 | 2,732 | 7,729 | 7,331 | 5,248 | 12,493 | 16,076 | 44,946 |
| 2021 | 7,028 | 3,230 | 14,253 | 12,076 | 6,690 | 16,841 | 31,557 | 56,767 |
| 2022 | 7,309 | 3,696 | 16,695 | 15,474 | 6,689 | 21,131 | 37,698 | 66,543 |
Note(s): The number of Form 10-Ks indicates the Form 10-K filings collected for the study from the U.S. Securities and Exchange Commission’s EDGAR database
Source(s): Table by authors
Table 10
Correlations between SDG-related ESG sentences and market capitalization by industry
| Industry | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| AGR | CON | FIR | MF | MN | RT | SRV | TCE | WT | Overall | |
| r | 0.36 | 0.29** | 0.56*** | 0.45*** | 0.44*** | 0.11 | 0.33*** | 0.34*** | 0.40*** | 0.42*** |
Note(s): r = Pearson correlation coefficient. Significance level a = 0.05; ***p < 0.001, **p < 0.01 (2-tailed). The natural logarithmic was taken for market capitalizations
Source(s): Table by authors
© Emerald Publishing Limited.