Word Embeddings and Machine Learning Classifiers Applications for Automatic Detection of Suicide Tendencies in Social Media

Abstract

This study presents an innovative and comprehensive model for the automatic detection of suicidal ideation in social media posts. Through an in-depth analysis of 50000 posts and the combination of four word embedding techniques (Word2Vec, GloVe, MPNet, and GPT-3) with five advanced classifiers, we have achieved an accuracy of over 90% in identifying users who may be at risk. Our results suggest that the integration of large language models like GPT-3 with deep neural network architectures offers a promising tool for suicide prevention in the digital realm, contributing to the development of automated screening systems capable of alerting mental health professionals to potential cases of risk.

INTRODUCTION

Identifying suicidal thoughts remains a formidable challenge in psychology, especially within the context of mental health disorders. According to the World Health Organization (WHO) [1], suicide claims approximately 703 000 lives annually worldwide. While this can affect people of any age, individuals between 15 and 29 are particularly vulnerable. WHO data from 2016 [2] reveals a global age-standardized suicide rate of 10.5 per 100  000, with significant variations across countries. High-income nations, for instance, report rates as high as 11.5 per 100  000. The pervasive stigma and societal taboos surrounding suicide hinder its detection in the general population. Fear of judgment often prevents those contemplating suicide from seeking help, making it difficult to identify at-risk individuals.

Securing relevant data is a primary hurdle in suicide identification. Some research endeavors circumvent this by extracting information from existing databases to construct focused datasets [3]. However, this method’s efficacy is constrained by its reliance on these data sources, limiting growth to instances where comparable data becomes publicly accessible. The internet’s ascendancy as a communication platform, facilitated by forums and social networks, has significantly amplified discussions about suicidal tendencies. This heightened online discourse offers unprecedented opportunities for large-scale analysis [4]. Research indicates that scrutinizing social networks can provide invaluable insights into individuals’ lives, including potential expressions of suicidal thoughts [5, 6].

Although infrequent, public posts about suicide can be found on social media platforms. A prime example is Reddit’s “r/SuicideWatch” forum, where users openly discuss suicidal tendencies. The anonymity afforded by such platforms empowers individuals to share personal details, including sensitive topics like healthcare, under assumed identities [7]. This environment may encourage those struggling with suicidal thoughts to seek solace and connection by expressing their inner turmoil through anonymous online channels.

To automate the detection of suicidal tendencies within text, classification algorithms rely on numerical text representations. These features, which encapsulate various textual aspects, render the text machine interpretable. Word Embeddings, a technique that generates feature representations based on semantic and syntactic relationships, have emerged as a cornerstone in this process. Tools like Word2Vec, GloVe, ELMo, and BERT effectively implement Word Embeddings [8]. By extracting rich semantic and syntactic information from text, word embeddings empower classification algorithms to accurately identify patterns indicative of suicidal tendencies.

This research aims to evaluate the efficacy of word embeddings in extracting relevant information from text for the purpose of detecting suicide tendencies in social media. By comparing the performance of various word embedding models in conjunction with different machine learning classifiers, this study seeks to identify the optimal combination for accurately identifying individuals at heightened risk of self-harm. Accurate detection is crucial for enabling timely interventions such as strengthening community mental health resources or developing specialized crisis resolution and home treatment services for high-risk populations.

RELATED WORK

Researchers and mental health professionals have long sought effective methods for identifying suicidal tendencies. As social media platforms emerge as potential sources of valuable data, it is imperative to build upon existing research. The fundamental components of suicide ideation detection encompass data acquisition, text representation, and classification. However, a primary obstacle in this field is the dearth of medically verified datasets.

Renjith’s research employed deep learning to identify suicidal ideation within social media posts [9]. Utilizing a Reddit dataset categorized into four risk levels, the study transformed text into numerical representations using Word2Vec embeddings pretrained on Google News. These embeddings were subsequently fed into various neural network architectures: support vector machines (SVM), convolutional neural networks (CNN), long short-term memory (LSTM), LSTM-CNN, and a combined LSTM-CNN-attention ensemble. The proposed ensemble model demonstrated superior performance, achieving an accuracy of 90.3%, precision of 91.6%, recall of 93.7%, and F1-score of 92.6% in detecting suicidal intentions.

Acuña’s research employed the Life Corpus, a bilingual dataset of 102 texts in English and Spanish [10]. To extract textual features, they utilized bag of words (BOW), bag of stems, bag of lemmas, bag of synsets, and bag of part-of-speech (POS) techniques. A comprehensive evaluation of 28 classification algorithms was conducted using Weka software, with performance assessed using F-measure, ROC Area, Precision, and Recall. The model achieved promising results, with an F1-score of 75% and ROC Area of 81% in differentiating between suicidal and nonsuicidal messages.

Metzler sought to categorize Twitter posts related to suicide to better grasp their influence on suicidal behavior and prevention strategies [11]. The study analyzed 3202 English tweets identified using suicide-related keywords. Textual data was transformed into numerical representations using TF-IDF, BERT, and XLNet. Both traditional (SVM) and cutting-edge (BERT, XLNet) classification models were employed. Model performance was assessed using accuracy, recall, precision, and F1-score. Deep learning models, especially BERT, significantly surpassed traditional methods, achieving 88% accuracy and 81% recall in classifying suicide-related tweets.

Ramirez sought to identify Twitter users at risk of suicidal ideation through a multimodal approach incorporating text, image, relational, and behavioral data [12]. Focusing on Spanish-speaking users, the study analyzed a dataset of 98 619 tweets using a two-tiered annotation process. Textual features included N-grams, word embeddings, and social, psychological, and behavioral attributes. Machine learning models (random forest, multilayer perceptron, logistic regression, SVM) were applied to classify users based on textual, visual, relational, and behavioral data. CNN was exclusively used for embedding generation. Evaluation metrics comprised precision, recall, F1-score, accuracy, and AUC. The multimodal approach, combining images, social, psychological, and textual features with random forest and multilayer perceptron, significantly outperformed baseline models, achieving an 88% recall score. CNN-based embeddings yielded recall scores between 81 and 87% depending on the data utilized.

Tadesse analyzed a Reddit dataset categorizing posts as suicidal or nonsuicidal [13]. The dataset comprised 3549 suicidal and 3652 nonsuicidal posts. Data was preprocessed to eliminate duplicates, tokenize, and remove irrelevant elements. Features included TF-IDF, bag of words, and statistical data, serving as a baseline alongside naïve Bayes, SVM, XGBoost, LSTM, and CNN classifiers. Word2Vec embeddings were integrated with LSTM and CNN. Evaluation metrics encompassed accuracy and F1 score. The proposed LSTM-CNN model excelled, achieving 93.8% accuracy, 93.4% F1 score, 94.1% recall, and 93.2% precision, surpassing baseline models.

Ji conducted an evaluation of three real-world datasets: UMD Reddit Suicidality, Reddit Suicide Watch and Mental Health, and a Twitter collection [14]. Textual features, including embeddings, sentiment, and topic, were extracted, and Relation Networks with Attention were employed to capture interconnections between elements. FastText, CNN, LSTM, RNN, and SSA classifiers were applied to categorize the data. Performance metrics encompassed accuracy, precision, recall, and F1-score. While Reddit datasets yielded scores between 50 and 66%, the Twitter dataset demonstrated superior performance with scores ranging from 80 to 84%. These findings suggest that the proposed method effectively improved classification accuracy compared to standard approaches.

DATA COLLECTION

The study involved compiling posts from repositories dedicated to suicide-related content. A primary challenge was the absence of medical validation for this data. To address this, researchers sourced data referenced in scholarly works or from reputable platforms like Hugging Face and Kaggle. The objective was to identify and assist individuals expressing suicidal thoughts in writing. Consequently, capturing complete sentence context was crucial. A total of 175 010 texts were gathered from a Hugging Face repository [15] comprising 77 223 suicidal and 97 787 nonsuicidal examples. Figure 1 outlines the data source and collection process. Detailed information about the origin and references can be found in the dataset card. For the experiment, a balanced subset of 50 000 texts (25 000 suicidal, 25 000 nonsuicidal) was selected, prioritizing shorter texts (100–300 characters).

Fig. 1. [Images not available. See PDF.]

Data source and distribution.

METHODOLOGY

This study aims to compare embeddings and classifiers for extracting textual meaning and developing a robust model for detecting suicide tendencies on social media. Word embeddings from open-source models (Word2Vec [16], GloVe [17], MPNet [18]) and commercial platforms (GPT-3 small and large [19], [20]) were employed to assign numerical values to words.

For classification, decision trees (DT), random forest (RF), logistic regression (LR), support vector machines (SVM), and multilayer perceptrons (MLP) from Scikit-Learn [21] were utilized. Hyperparameter tuning via grid search optimized classifier performance.

Data Preprocessing

Preprocessing a corpus–a substantial collection of text–is crucial in natural language processing (NLP) to transform raw data into a suitable computational format. The Freeling library [20–24] was employed for this purpose, providing tools for morphological analysis, named entity recognition, and part-of-speech tagging. The preprocessing pipeline included removing extraneous text, tokenization, and stop word elimination. While there are several methods to explore sentence similarity [27], the sentence similarity removal was assessed using cosine similarity. Text cleaning involved eliminating punctuation and stop words to enhance focus on meaningful terms and ensure consistent formatting.

Text Embeddings

Preprocessed text was transformed into numerical representations using word embedding techniques to capture semantic and syntactic word relationships. Word2Vec, GloVe, and MPNet were employed, with Word2Vec (pretrained with “word2vec-google-news-300” [28]) and GloVe using 300-dimensional vectors. The HuggingFace “all-mpnet-base-v2” model provided 768-dimensional MPNet embeddings. Additionally, GPT-3’s text-embedding-3-small (1536 dimensions) and text-embedding-3-large (3072 dimensions) were explored for advanced text representation. Sentence embeddings were calculated by averaging word vectors.

Classification Algorithms

Automatic classification algorithms were employed to identify indicators of suicidal tendencies within social media posts. These algorithms assigned probabilities to user-generated text expressing suicidal thoughts. Five algorithms were utilized: decision trees, random forest, logistic regression, support vector machine, and multilayer perceptron. Data was divided into training (75%) and testing (25%) sets. Model performance was evaluated using accuracy, precision, recall, and F1-score, with a particular emphasis on recall due to the critical nature of correctly identifying suicidal tendencies.

RESULTS

The experiment consisted of 25 unique combinations of embeddings and classification algorithms, outlined in Table 1. The goal was to assess the accuracy of each combination in identifying suicidal tendencies. Classifiers generated two output labels: “Possible suicidal tendencies” (1) and “No suicidal tendencies” (0). Performance metrics, including accuracy, precision, recall, and F1-score, were calculated for the suicidal class and visualized for each embedding. Hyperparameters were tuned for each embedding-model pair to optimize performance. Table 2 details the hyperparameter configurations.

Table 1. . Cross-validations scores in embeddings–classifiers combinations

Models	DT	RF	LR	SVM	MLP
Word2Vec	0.72	0.79	0.82	0.65	0.83
GloVe	0.68	0.76	0.81	0.61	0.81
MPNet	0.74	0.82	0.86	0.71	0.86
GPT-3-Small	0.75	0.84	0.88	0.73	0.89
GPT-3-Large	0.75	0.85	0.88	0.77	0.90

Table 2. . Hyperparameters implemented by embedding–classifier

Models	Hyperparameters used
GloVe - DT	'criterion': 'entropy', 'max_depth': 500, 'max_features': None, 'min_samples_leaf': 61, 'min_samples_split': 88, 'splitter': 'best'
GloVe - LR	'solver': 'liblinear', 'penalty': 'l2', 'max_iter': 100
GloVe - MLP	'activation': 'identity', 'alpha': 0.012834151269284711, 'hidden_layer_sizes': [50, 30], 'learning_rate': 'invscaling', 'max_iter': 300, 'solver': 'lbfgs'
GloVe - RF	'criterion': 'entropy', 'max_depth': None, 'max_features': None, 'min_samples_leaf': 8, 'min_samples_split': 17, 'n_estimators': 300
GloVe - SVM	'max_iter': 900, 'kernel': 'sigmoid', 'gamma': 'auto', 'degree': 5
GPT-3 - DT	'criterion': 'gini', 'max_depth': None, 'max_features': None, 'min_samples_leaf': 53, 'min_samples_split': 71, 'splitter': 'best'
GPT-3 - LR	'solver': 'newton-cg', 'penalty': 'l2', 'max_iter': 700
GPT-3 - MLP	'activation': 'relu', 'alpha': 0.004390090923029554, 'hidden_layer_sizes': [1500, 600], 'learning_rate': 'adaptive', 'max_iter': 400, 'solver': 'lbfgs'
GPT-3 - RF	'criterion': 'entropy', 'max_depth': 500, 'max_features': 'log2', 'min_samples_leaf': 18, 'min_samples_split': 85, 'n_estimators': 300
GPT-3 - SVM	'max_iter': 700, 'kernel': 'rbf', 'gamma': 'auto', 'degree': 4
GPT-3-L - DT	'criterion': 'entropy', 'max_depth': 500, 'max_features': None, 'min_samples_leaf': 61, 'min_samples_split': 88, 'splitter': 'best'
GPT-3-L - LR	'solver': 'newton-cg', 'penalty': 'l2', 'max_iter': 700
GPT-3-L - MLP	'activation': 'relu', 'alpha': 0.004390090923029554, 'hidden_layer_sizes': [1500, 600], 'learning_rate': 'adaptive', 'max_iter': 400, 'solver': 'lbfgs'
GPT-3-L - RF	'criterion': 'entropy', 'max_depth': 300, 'max_features': 'sqrt', 'min_samples_leaf': 70, 'min_samples_split': 29, 'n_estimators': 100
GPT-3-L - SVM	'max_iter': 900, 'kernel': 'sigmoid', 'gamma': 'auto', 'degree': 5
MPNet - DT	'criterion': 'gini', 'max_depth': None, 'max_features': None, 'min_samples_leaf': 53, 'min_samples_split': 71, 'splitter': 'best'
MPNet - LR	'solver': 'sag', 'penalty': 'l2', 'max_iter': 900
MPNet - MLP	'activation': 'relu', 'alpha': 0.004390090923029554, 'hidden_layer_sizes': [1500, 600], 'learning_rate': 'adaptive', 'max_iter': 400, 'solver': 'lbfgs'
MPNet - RF	'criterion': 'entropy', 'max_depth': 500, 'max_features': 'log2', 'min_samples_leaf': 18, 'min_samples_split': 85, 'n_estimators': 300
MPNet - SVM	'max_iter': 700, 'kernel': 'rbf', 'gamma': 'auto', 'degree': 4
Word2vec - DT	'criterion': 'entropy', 'max_depth': 500, 'max_features': None, 'min_samples_leaf': 61, 'min_samples_split': 88, 'splitter': 'best'
Word2vec - LR	'solver': 'newton-cholesky', 'penalty': 'l2', 'max_iter': 100
Word2vec - MLP	'activation': 'relu', 'alpha': 0.004390090923029554, 'hidden_layer_sizes': [1500, 600], 'learning_rate': 'adaptive', 'max_iter': 400, 'solver': 'lbfgs'
Word2vec - RF	'criterion': 'entropy', 'max_depth': 500, 'max_features': 'log2', 'min_samples_leaf': 18, 'min_samples_split': 85, 'n_estimators': 300
Word2vec - SVM	'max_iter': 900, 'kernel': 'sigmoid', 'gamma': 'auto', 'degree': 5

Figure 1 illustrates Word2Vec results, indicating logistic regression (LR), multilayer perceptron (MLP), and random forest (RF) as top performers with balanced metrics (79–84%). decision trees (DT) achieved a balanced average of 71.5%, while Support vector machine (SVM) exhibited lower precision (62%) and recall (78%). Notably, SVM demonstrated reasonable recall in identifying the suicidal class. Figure 2 presents GloVe embedding results, showing similar trends to Word2Vec with minor score variations (1–2%). These differences might be attributed to the underlying training data, with Google’s dataset potentially offering advantages.

Fig. 2. [Images not available. See PDF.]

Scores obtained using Word2vec embeddings.

Figure 3 presents MPNet embedding results, indicating logistic regression (LR) and multilayer perceptron (MLP) as top performers with scores between 85 and 86%, outperforming Word2Vec and GloVe. Decision trees (DT) and support vector machines (SVM) showed lower performance across all metrics, failing to reach 80%. Random forest (RF) exhibited mixed results, with recall at 77% but exceeding 80% in accuracy, precision, and F1-score.

Fig. 3. [Images not available. See PDF.]

Scores obtained using GloVe embeddings.

In Figs. 4 and 5 demonstrate the superior and consistent performance of GPT-3 embeddings across all models (LR, MLP, DT, RF, SVM). GPT-3-Small achieved peak performance with MLP (89% across all metrics) and strong results with LR (88%). GPT-3-Large surpassed all other embeddings, with MLP reaching 90% (91% recall) and LR at 89%. While SVM showed a 12% precision drop compared to GPT-3-Small, recall improved by 12%. Larger embeddings may introduce challenges, including lower scores and longer training times for certain models.

Fig. 4. [Images not available. See PDF.]

Scores obtained using MPNet embeddings.

Fig. 5. [Images not available. See PDF.]

Scores obtained using GPT-3-Small embeddings.

Only combinations achieving a 10-fold cross-validation score above 85% were selected for further analysis, ensuring model reliability. Figure 6 presents seven qualifying combinations with recall scores ranging from 82 to 91%. Overall performance was balanced, with minor inconsistencies in GPT-3-L and random forest (RF).

Fig. 6. [Images not available. See PDF.]

Scores obtained using GPT-3-Large embeddings.

Fig. 7. [Images not available. See PDF.]

Top combinations based on cross-validation score, threshold +85%.

High-dimensional embeddings, particularly GPT-3-Small and GPT-3-Large, demonstrated exceptional performance in classifying suicidal tendencies. GPT-3 embeddings consistently outperformed others, capturing rich contextual information. Multilayer perceptron (MLP) and logistic regression (LR) classifiers excelled when paired with these embeddings. The optimal combination, GPT-3-Large with MLP, achieved outstanding results with 90% accuracy, precision, recall, and F1-score, emphasizing MLP’s efficiency in handling high-dimensional data.

Additionally, GPT-3-Small offers a cost-effective alternative to larger embeddings while delivering respectable results. MPNet, while incurring no monetary expense, also demonstrated strong performance, achieving scores of approximately 85% across all metrics.

CONCLUSIONS

A comprehensive series of experiments was undertaken to assess the efficacy of diverse embedding-classifier combinations in detecting suicidal tendencies. An initial dataset of 50 000 texts was curated from academic research, internet databases, and repositories. Preprocessing steps, including stop word removal and punctuation elimination, were applied to the text corpus. Subsequently, five distinct embedding models were employed to transform text into numerical representations, each capturing textual context at varying dimensional levels. Traditional classification algorithms were then trained on these embeddings to perform the classification task.

The findings indicate that GPT-3 embeddings, specifically GPT-3-Large, achieved superior performance across all metrics (90% accuracy, recall, precision, and F1-score) when paired with an MLP model. While demonstrating strong compatibility with other classifiers, particularly logistic regression, GPT-3 embeddings consistently outperformed alternatives. MPNet embeddings exhibited moderately better results than Word2Vec and GloVe, with logistic regression and multilayer perceptron models achieving scores between 85 and 86%. Word2Vec and GloVe demonstrated comparable performance, ranging from 79 to 84% when combined with logistic regression, multilayer perceptron, and random forest classifiers

Overall, multilayer perceptron (MLP) demonstrated superior performance across all embedding models, achieving scores between 83 and 91% for accuracy, precision, recall, and F1-score. Logistic regression (LR) exhibited comparable results, ranging from 82 to 89%. Decision trees (DT) produced moderate outcomes, with average scores of 71.5% on Word2Vec and lower performance on GloVe. Random forest (RF) displayed inconsistent results, excelling in accuracy, precision, and F1-score (80%+) with high-dimensional embeddings but showing lower recall (77%). Support vector machines (SVM) consistently underperformed, with particularly low precision (60% on GloVe) and instability across metrics. SVM’s only notable result was a recall score of 89% with GPT-3-Small embeddings.

The study’s findings underscore the critical role of text numeric representation in classifier performance. Multilayer perceptron (MLP) models demonstrated exceptional capability in predicting suicidal tendencies, achieving accuracy, precision, recall, and F1-scores exceeding 90% without demanding extensive computational resources. While less expensive or free embedding options yielded commendable results (85–89% across metrics), the study suggests that larger embeddings excel in creating detailed text representations, leading to superior classification outcomes.

FUTURE WORK

This study explored the efficacy of traditional classification algorithms for detecting suicidal tendencies without relying on computationally intensive deep models. Future research could delve into the potential of deep neural networks for this task. Two primary outcomes are anticipated:

(1) Performance enhancement: Deep neural networks might surpass the performance of traditional models, achieving accuracy, precision, recall, and F1-scores exceeding 90%.

(2) Comparable results: Alternatively, deep models may produce results similar to those obtained in this study, suggesting the adequacy of traditional methods for this specific task.

A promising avenue for future research involves expanding the corpus by analyzing and categorizing text from social media platforms. Given the scarcity of publicly available, medically validated data related to suicidal tendencies, a practical approach could involve utilizing social media platforms as a data source. By employing suicide-related keywords to extract relevant content, researchers can significantly augment the dataset and potentially enhance the accuracy of suicide tendencies detection models. A larger, more diverse dataset is expected to improve model performance and facilitate more effective identification of suicidal content.

FUNDING

This work was partially supported by the Government of Mexico through CONAHCYT (master scholarship and SNII) and Tecnológico Nacional de México (TecNM) CENIDET (project titled “Identification of possible suicide tendencies in social networks messages through machine learning models and natural language processing” accepted in Scientific Research Project, Technological and Innovative Development Call from TecNM 2024).

CONFLICT OF INTEREST

The authors of this work declare that they have no conflicts of interest.

Publisher’s Note.

Pleiades Publishing remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

AI tools may have been used in the translation or editing of this article.

REFERENCES

1 WHO, “Suicidio,” World Health Organization.

2 WHO, “One in 100 deaths is by suicide,” World Health Organization.

3 Rodríguez-Ruiz, J.G.; Galván-Tejada, C.E.; Vázquez-Reyes, S.; Galván-Tejada, J.I.; Gamboa-Rosales, H. Classification of depressive episodes using nighttime data; a multivariate and univariate analysis. Program. Comput. Software; 2020; 46, pp. 689-698. [DOI: https://dx.doi.org/10.1134/S0361768820080198] 1425.94018

4 Luo, J.; Du, J.; Tao, C.; Xu, H.; Zhang, Y. Exploring temporal suicidal behavior patterns on social media: Insight from Twitter analytics. Health Inf. J.; 2020; 26, pp. 738-752. [DOI: https://dx.doi.org/10.1177/1460458219832043]

5 Sierra, G. Suicide risk factors: A language analysis approach in social media. J. Lang. Soc. Psychol.; 2022; 41, pp. 312-330. [DOI: https://dx.doi.org/10.1177/0261927X211036171] 07619697

6 Ji, S.; Yu, C.P.; Fung, S.; Pan, S.; Long, G. Supervised learning for suicidal ideation detection in online user content. Complexity; 2018; 2018, pp. 1-10. [DOI: https://dx.doi.org/10.1155/2018/6157249]

7 Ma, X., Hancock, J., and Naaman, M., Anonymity, intimacy and self-disclosure in social media, in Proc. Conf. on Human Factors in Computing Systems, Association for Computing Machinery, 2016, pp. 3857–3869. https://doi.org/10.1145/2858036.2858414

8 Castro Sánchez, N.A. Procesamiento de diccionarios en la linguistica computacional; 2021;

9 Renjith, S.; Abraham, A.; Jyothi, S.B.; Chandran, L.; Thomson, J. An ensemble deep learning technique for detecting suicidal ideation from posts in social media platforms. J. King Saud Univ., Comput. Inf. Sci.; 2022; 34, pp. 9564-9575. [DOI: https://dx.doi.org/10.1016/j.jksuci.2021.11.010]

10 Acuña Caicedo, R.W., Gómez Soriano, J.M., and Melgar Sasieta, H.A., Assessment of supervised classifiers for the task of detecting messages with suicidal ideation, Heliyon, 2020, vol. 6, no. 8, p. e04412. https://doi.org/10.1016/j.heliyon.2020.e04412

11 Metzler, H., Baginski, H., Niederkrotenthaler, T., and Garcia, D., Detecting potentially harmful and protective suicide-related content on Twitter: machine learning approach, J. Med. Internet Res., 2022, vol. 24, no. 8. https://doi.org/10.2196/34705

12 Ramírez-Cifuentes, D., et al., Detection of suicidal ideation on social media: Multimodal, relational, and behavioral analysis, J. Med. Internet Res., 2020, vol. 22, no. 7. https://doi.org/10.2196/17758

13 Tadesse, M.M., Lin, H., and Yang, L., Detection of suicide ideation in social media forums using deep learning, Algorithms, 2020, vol. 13, no. 1. https://doi.org/10.3390/a13010007

14 Ji, S.; Li, X.; Huang, Z.; Cambria, E. Suicidal ideation and mental disorder detection with attentive relation networks. Neural Comput. Appl.; 2022; 34, pp. 10309-10319. [DOI: https://dx.doi.org/10.1007/s00521-021-06208-y] 1530.93458

15 Joheras, PrevenIA/spanish-suicide-intent Datasets at Hugging Face. https://huggingface.co/datasets/PrevenIA/spanish-suicide-intent.

16 Mikolov, T., Chen, K., Corrado, G., and Dean, J., Efficient estimation of word representations in vector space, 2013. arXiv:1301.3781, 2013.

17 Pennington, J., Socher, R., and Manning, C.D., Glove: global vectors for word representation, Proc. Conf. on Empirical Methods in Natural Language Processing (EMNLP), Doha, 2014, pp. 1532–1543.

18 Song, K.; Tan, X.; Qin, T.; Lu, J.; Liu, T.-Y. Mpnet: masked and permuted pre-training for language understanding. Adv. Neural Inf. Process. Syst.; 2020; 33, pp. 16857-16867.1530.65093

19 OpenAI, Embeddings Platform. https://platform.openai.com/docs/guides/embeddings.

20 OpenAI, Embeddings. https://openai.com/index/new-embedding-models-and-api-updates.

21 Pedregosa, F. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res.; 2011; 12, pp. 2825-2830.28543481280.68189

22 Carreras, X., Chao, I., Padró, L., and Padró, M., FreeLing: An open-source suite of language analyzers, Proc. 4th Int. Conf. on Language Resources and Evaluation (LREC’04), Lisbon, 2004.

23 Padró, L. Analizadores multilingues en FreeLing. Linguamatica; 2011; 3, pp. 13-20.

24 Padró, L. and Stanilovsky, E., FreeLing 3.0: Towards wider multilinguality, in Proc. Language Resources and Evaluation Conf. (LREC 2012), Istanbul, May 2012.

25 Padró, L., Collado, M., Reese, S., Lloberes, M., and Castellón, I., FreeLing 2.1: Five years of open-source language processing tools, Proc. 7th Language Resources and Evaluation Conference (LREC’10), La Valletta, May 2010.

26 Atserias, J., Casas, B., Comelles, E., González, M., Padró, L., and Padró, M., FreeLing 1.3: Syntactic and semantic services in an open-source NLP library, Proc. 5th Int. Conf. on Language Resources and Evaluation (LREC 2006), Genoa, May 2006.

27 Burdonov, I.; Maksimov, A. Twenty similarity functions for two finite sequences. Program. Comput. Software; 2023; 49, pp. 373-387.4652354[DOI: https://dx.doi.org/10.1134/S0361768823050031] 1543.68475

28 Google, Word2Vec – Google News. https://code.google.com/archive/p/word2vec.