Content area
This paper extends an initial investigation of eHealth from the developers’ perspective. In this extension, our focus is on mobile health data. Despite the significant potential of this development area, few studies try to understand the challenges faced by these professionals. This perspective is relevant to identify the most used technologies and future perspectives for research investigation. Using a KDD-based process, this work analyzed eHealth and mHealth discussions from Stack Overflow (SO) to comprehend this developers’ community. We got and processed 6082 eHealth and 1832 mHealth questions. The most discussed topics include manipulating medical images, electronic health records with the HL7 standard, and frameworks to support mobile health (mHealth) development. Concerning the challenges faced by these developers, there is a lack of understanding of the DICOM and HL7 standards, the absence of data repositories for testing, and the monitoring of health data in the background using mobile and wearable devices. Our results also indicate that discussions have grown mainly on mHealth, primarily due to monitoring health data through wearables and about how to optimize resource consumption during health-monitoring.
Introduction
Currently, almost 70% of the world’s population has access to smartphones (this represents 5.31 billion unique users). This percentage is growing with the inclusion of 95 million of new mobile users in 2021.1 These numbers show how our society has focused its digital experience (services, entertainment, games, news, social media, among others) on devices capable of accompanying the user throughout the day. This trend could not be different with health services.
Since the definition of the term eHealth (predecessor of the term mHealth - mobile health) in 2001 [1], this research area has been strengthened as a more significant population-share turns their attention to well-being and health issues [2]. Furthermore, the increase of computational paradigms such as the Internet of Things (IoT) also contributed to eHealth strengthening. In IoT, for example, things like smartwatches can monitor an elderly in his/her house, sending relevant information to physicians for improving user care [3]. Also, there is a large number of initiatives in digital image processing [4], electronic health records [5], mobile health [6], and machine learning applied on health [7, 8].
These initiatives usually result in new technologies with direct benefits for their users’ quality of life [9], and their development creates many other valuable data such as the crowd knowledge built from technical discussions among practitioners working in this area [10]. This knowledge is helpful in many ways, whether to boost the development by overcoming issues already faced by other developers [11], or even to highlight demands to be addressed by researchers [12]. Unfortunately, this crowd knowledge is often diffused in different software repositories [13], and it needs to be systematically mined to become intelligible.
Among these software repositories, the Q &A (Question & Answer) websites have a significant relevance due to their use by practitioners to discuss technologies and strategies to solve programming challenges [14, 15]. A striking Q &A website is the Stack Overflow2 (SO), which was founded in 2008 and had one of the largest technology enthusiasts communities. Stack Overflow has over 14 million discussions and 50 million monthly visitors. Also, SO data is public and can be accessed through the Stack Exchange Data Explorer tool.3 In addition to being a widely used tool by technology professionals, SO is significantly used for scientific studies [11, 14–16]. We can highlight the studies from [17] and [15], which were conducted to understand the types of questions asked by the developers, and from [18, 19] that observed how the SO code snippets are used.
Despite the relevance and applicability of the knowledge found on these websites by Information and Communications Technology (ICT) professionals (developers, analysts, and software engineers) who work directly in the eHealth area, we have not seen studies aiming to systematically mine the knowledge that emerges from the discussion of eHealth topics in Q &A websites. In the literature, most studies have used public databases (e.g., social networks, and paper repositories) to predict trends from the perspective of eHealth end-users [20–23], or from the researcher’s point of view [24–27].
In this way, we published the first work with an exploratory analysis of the eHealth Stack Overflow discussions aiming to understand the eHealth development community behavior considering the developers’ perspective [28]. Now, this paper expands the initial work by focusing on mobile health data. Thus, in addition to updating the original dataset, this study provides an in-depth discussion of the practical challenges that mHealth developers face, offering possible solutions and pointing out paths for future investigations.
Four Research Questions (RQ) were designed to guide our study. The first three Research Questions come from the initial study and provide an overview of the technologies, essential subjects, and types of questions asked by eHealth developers. The fourth question (RQ4) was included in this extension and sought to discuss the challenges and future visions for mobile health.
RQ1: What technologies have been discussed in eHealth?
Rationale: This question is essential to analyzing the software engineering artifacts used by eHealth developers. This information can assist practitioners in choosing the most suitable technologies for new eHealth projects.
RQ2: What eHealth subjects have been discussed in the Stack Overflow?
Rationale: Our interest in this question is to find hot topics related to eHealth discussions. Moreover, these topics can be used as a taxonomy to organize the eHealth knowledge present in the Stack Overflow.
RQ3: What types of questions asked by developers related to eHealth are most recurrent on Stack Overflow?
Rationale: This question aims to investigate what kind of questions are demanding more attention in this community. These demands can point out interesting challenges for future studies.
RQ4: What are the challenges and future visions for mobile health from developers’ perspective?
Rationale: Our interest with this question is to deepen the discussion on mHealth data, presenting the main challenges faced by developers and possible solutions for them. In addition, this discussion favors identifying future research paths to optimize mHealth solutions.
DICOM, EHR, and mHealth are the major subjects discussed by developers
Google Fit and HealthKit launch increased 250% mHealth discussions
Discussions about battery optimization in mHealth have more than 74k views
Health monitoring using wearables has been growing
There is a firm need for testing data in all eHealth subareas
Background monitoring that focuses on optimizing resources is a priority
There is a gap related to making health APIs easier
Background
EHealth has the potential to reduce costs and improve the quality of healthcare services. However, there are still open development challenges like high availability, scalability, fault tolerance, data management, interoperability, security and privacy, and user experience [29–31]. This article aims to analyze eHealth trends and demands using Stack Overflow discussions and, consequently, observe how developers are dealing with these challenges, especially the challenges related to the development of mobile health solutions.
This section will present concepts and techniques used in this work, such as mobile health systems, what Stack Overflow is, how it is used, and how the Topic Modeling technique works.
Mobile Health
Mobile health (mHealth) is defined by the World Health Organization (WHO) as the delivery of medical practice by mobile devices, including smartphones, tablets, or wearable monitoring devices [32]. The mHealth apps are primarily designed for mobile devices and are generally programmed to provide their users with warnings and advice about health conditions. They also facilitate the collection and sharing of health data and the delivery of health services [33]. Using mobile technologies to deliver health services can change how its users engage with Healthcare Providers (HCP) and their care.
The mHealth applications can be used in a variety of contexts and to assist in the treatment of various diseases, including diabetes [34, 35], high blood pressure [36, 37], cardiovascular disease [38], lung diseases [39], cancer [40] among other diseases [41, 42]. Over the years, the number of mobile health systems has gradually increased [27, 28]. Furthermore, according to Ming et al. (2020) , with the outbreak of the COVID-19 pandemic [44], there was an even more significant increase in this kind of application in the last three years.
According to Paiva et al. [27], we can relate the mobile applications to the classification of WHO Digital Health Initiatives (DHI). This classification represents a WHO initiative to classify the use of mobile technologies to support health systems [45]. DHI has four macro classes: (i) Interventions for Clients, which include both systems aimed at healthcare users and caregivers of clients using healthcare services; (ii) Interventions for Health Providers with solutions focused on those who provide health services; (iii) Interventions for Health Systems Managers with solutions that involve the management of public health systems, which can generate alerts and assist in decision making; (iv) Interventions for Data Services, which consists of interventions related to the collection, management, use and exchange of data.
Most mHealth systems are related to Personal Health Tracking, including apps where customers use smartphone sensors, health records, and wearables to monitor their health. On the other hand, many mHealth apps include Interventions for Healthcare Providers, including Telemedicine apps that offer remote healthcare delivery. In addition, applications that work with interventions for data services focus on data collection, management, and use category. However, the authors of [27] did not identify any studies until the beginning of 2020 presenting mHealth systems aimed at Interventions for Health Systems Managers and Interventions for Data Services.
Some studies present propositions of software artifacts for the development of mHealth systems, including models and frameworks [46, 47]. However, in our previous study [28], we found evidence that developers seem to be still unfamiliar with mHealth development software artifacts.
Stack Overflow
Regarding Stack Overflow, we decided to focus on this database because of its strength and representativeness for the software development community [15]. Currently, the SO has more than 10 million users around the world.4 Figure 1 presents the essential elements of the questions and answers on this platform.
Fig. 1 [Images not available. See PDF.]
Q &A structure in Stack Overflow [48]
The title and the body are the first elements of the question. They contain information about the problem faced by the user. In addition to these elements, the questions have tags used to categorize them and assist in the retrieval of the information process. A question can have several answers provided by different users (in the question presented in Fig. 1, there were 20 answer posts). The user who asked the question (the author) can accept one of the answers. Although it cannot be considered the best answer, the accepted answer usually represents the most suitable solution considering the author’s opinion. Both questions and answers have a score computed by the difference between the up-votes and down-votes. In this work, we consider the question score to filter our dataset because it measures the relevance of a post to the community.
We used the discussion definition proposed by [49]: “a combination of a question and one (or more) answers, where there is at least one answer whose author is not the author of the questions” to filter our target questions in SO. This definition excludes questions that have not received answers and questions that all answers were provided by the same user who asked the question. The rationale for using this definition is that we understand that unanswered questions or questions with answers provided only by the question’s author cannot represent relevant topics for the development community.
Stack Overflow data also have been used as income for many scientific works [11, 14–19]. In addition, some researchers have investigated why the questions are asked or the kind of information requested in the questions. Regarding these characteristics, [14] proposed one of the most adopted taxonomy to define the question type. It has ten categories: how-to, for questions that ask for instructions; discrepancy, for questions that present unexpected results; environment, for questions related to development environments; error, to questions that include a specific error message; decision help, for questions that seek opinions about aspects of the development; conceptual, for abstract questions related to the concepts of the area; review, for questions that request the review of code snippets; non-functional, for questions about non-functional requirements; novice, for questions in which it is clear that novice developers asked them; and, finally, noise, for questions not related to the problem investigated.
In this work, we decided to adopt this taxonomy because (i) it covers many different question types, such as questions about how to solve a problem related to decision-making; (ii) it helps to understand better the question nature asked by eHealth developers; and, iii) it is suitable for different contexts [15].
Topic Modeling
The topic modeling problem can be defined as identifying semantically coherent word sets (topics) to represent a textual data source and, hence, get insights about it [50]. We decided to use the Latent Dirichlet Allocation (LDA) algorithm [51] as our topic modeling strategy. This choice was based on the need to cluster the most critical terms in a corpus of documents. Also, several papers refer to LDA as the most used algorithm for this type of study [52, 53].
LDA uses a Bayesian model to calculate the probability of each topic concerning the documents and the probability of terms to topics. It initializes by creating a random relationship of each word in a document with one of the initial k topics defined by the user [51]. As a result, the LDA provides k topics composed of a set of words that need to be interpreted by experts to consolidate their meaning.
Fig. 2 [Images not available. See PDF.]
LDAvis screenshot for mHealth posts with 15 topics
It is common to use data visualization techniques [54] to interpret the resulting topics. In this work, we chose the LDAvis method, an interactive, web-based visualization method that helps users interpret the topic-term relationships in an LDA model [54]. The relevance of this visualization method lies in the difficulty of interpreting the results from the Latent Dirichlet Allocation algorithm [55]. LDAvis has two panels (left and right) (Fig. 2). The left panel has a two-dimensional distribution of topics. The inter-topic distance calculates the position of the centroids, and the topic’s prevalence is mapped in the circles’ area. The right panel presents the most significant terms for each topic. In this panel, it is also possible to adjust the relevance of a term to a topic, helping to interpret the topic’s semantics.
N-Gram Data Analysis
Several techniques can be used to analyze textual data. Among these techniques, a simple and widely used model is called N-gram Data Analysis [56]. N-gram consists of continuous text strings of length n. Thus, it is possible to collect the frequency of expressions in a dataset. In the literature, this technique is used in sentiment classification [57], online detection of fake news [58], and fault location in software [59]. We decided to use this technique to analyze the dataset of questions focused on mobile health to find more frequent expressions in this work.
Study Design
This paper expands on an empirical study focused on the discussions of the eHealth developers’ community [28]. We decided to deepen our analysis of the mobile health data in this new study, considering the developer’s perspective. The rationale for this decision regards the current relevance of the mHealth area [60].
Considering this goal, our study design was defined based on Knowledge Discovery in Databases (KDD) process [61]. KDD was chosen given the need to extract knowledge from an unstructured database (Stack Overflow website). Then, guidelines for data mining studies in software repositories were considered to adapt this process for evidence-based software engineering [13]. The following subsections detail the phases of our design study, which are illustrated in Fig. 3.
Fig. 3 [Images not available. See PDF.]
Our study design
Selection
One of the first and most important data mining tasks is identifying one or more data sources that can provide insights to answer the Research Questions. Regarding data mining in software engineering—also known as mining software repository—these data sources are generally software repositories that store, for example, source code, binary code, version control data, configuration files, execution logs, execution traces, or communication between developers [13]. In this work, we selected the English version of Stack Overflow as the data source due to its relevance to researchers and technology professionals [15].
After the data source definition, we defined the search strategy. This activity required many trials to find a suitable strategy because (i) it was not possible to use the main Stack Overflow search field (that considers the title and questions’ body) since it can return inconsistent data; (ii) the terms commonly used in the literature as synonyms for eHealth (according to MeSH controlled vocabulary5) did not return any data. This vocabulary associates eHealth to mobile health, mHealth, telehealth, and telemedicine terms; (iii) and there is no specific tag that characterizes eHealth questions.
After a comprehensive analysis, we realized a divergence between the MeSH terms and terms practitioners use to categorize eHealth issues in Stack Overflow. For example, the question “How is it possible that Google Fit app measures number of steps all the time without draining the battery?”—that should be classified as mHealth in MeSH—was tagged with the technology (Google Fit) and the platform (Android).
We also used other tags defined by our experience, but they did not return posts focused on eHealth. For example, we tried the following tags: electronic health record software, hospital management software, urgent care apps, fall, blood, mysignals, ecg, emg, body, glucose, oximeter, elderly, older, adult, fitness, treatment, caregiver, healthcare, care, hospital, disease, health-monitoring, kubernates-health-check, and fiware-health.
Thus, after this empirical analysis, we decided to use the term “health” as the starting point to execute a snowballing [62] search for tags. Initially, we searched for all tags related to the term “health” using Stack Overflow’s tag search system.6 Then, for each tag found, we analyzed its definition to check if it is related to eHealth. The selected tags are shown in Table 1.
Table 1. Tags used to query the target data
Subareas | Tags | Posts |
|---|---|---|
DICOM | Dicom, pydicom, fo-dicom, evil-dicom, dicomweb, niftynet, dcm4che | 1418 |
EHR | Healthvault, intersystems-healthshare, hl7-fhir, ccd, cda, hl7, hapi, mirth, mirth-connect, hl7-v2, hl7-v3, hl7-cda, c-cda, hapi-fhir, fhir-server-for-azure, fhir-net-api, smart-on-fhir, openehr, nhapi, dstu2-fhir, hapi-fhir-android-library, snomed-ct, mdht, btahl7, medical, icd, hit | 2092 |
mHealth | Health-kit, hkhealthstore, google-health, samsung-health, flutter-health, mapmyfitness, withings, heartrate, fitbit, hksamplequery, hkobserverquery, pedometer, google-fit, researchkit, samsung-gear-fit, strava, google-fit-sdk, wearables, hapi-fhir-android-library, carekit, medical, icd, hit | 2572 |
Total of Questions | 6082 | |
Total of Questions without Duplicates | 5608 | |
In the first part of this study, we found 6082 questions using the selected tags (without date filters), and we got 5608 after duplicate removal. This query was performed on December 10, 2019. Figure 4 summarizes the activities executed in this phase.
Fig. 4 [Images not available. See PDF.]
Activities of the selection phase
The tags presented in Table 1 were classified into three subareas: DICOM, to represent questions about digital medical image processing; EHR, for issues focusing on Electronic Health Records; and mHealth, for questions about the development of Mobile Health applications. The authors proposed this classification following the tags’ similarity, their community definitions, and the results of an LDA pilot study performed after removing duplicate questions. For sure, these are not the only subareas in eHealth. However, the tags found indicate that these three are the most discussed by developers on SO.
Fig. 5 [Images not available. See PDF.]
LDA model used to classify the tags and questions
In the pilot study, we built several LDA models with different topic numbers. Analyzing the model with three topics, a clear division among the questions in these subareas was observed. Figure 5 brings the three topic models and the ten most recurrent terms for each one.
In the second part of the study, referred to as the study extension, we used only tags classified as mHealth to retrieve 1,832 questions. Among these issues, we found and removed 16 duplicates. Thus, 1,816 mHealth questions were left. This query was performed on April 4, 2022.
Processing
After data collection, we performed three pre-processing activities. First, we carried out a filter considering the question popularity and the discussion definition [49]. For the popularity filter, we used the third quartile of the question score. This filter’s rationale is to reduce the noise that can be included by less popular questions or questions that only one user sent answers [63].
We also adopted the third quartile to avoid bias regarding the definition of a hard threshold. Hence, our analyses consider the community’s most relevant questions. These filters reduced the number of questions from 5608 to 1112, implying a considerable data reduction. However, we decided to continue as our focus was on the most relevant questions using criteria defined by the eHealth developer community.
In the second activity, we performed a manual analysis to get valuable data from each discussion. This process is a labor-intensive task, and it was conducted by two researchers with meetings to do the agreement check. This manual analysis was necessary because there is no automated method for classifying the question type, and this information is essential to our three initial research questions. For this classification, we decided to use the well-known taxonomy proposed by [14] due to its characteristic of being able to be used in different study areas. This taxonomy has ten different types: how-to, discrepancy, environment, error, decision help, conceptual, review, non-functional, novice, and noise. For each discussion, we also did an open coding [64] to identify developers’ concerns.
This manual analysis reduced the number from 1,112 to 1,076 due to the classification of 36 questions as noise (questions not related to eHealth but tagged with any of the tags defined in Table 1).
Here, it is essential to mention that we decided not to apply the discussion and popularity filters to the updated mHealth data in expanding the study. This decision lies in the fact that we were seeking, in Research Question 4, to understand the most faced challenges, even if these are not the most popular within the community. This different perspective allows us to complement the first analysis, pointing out future paths for the mHealth area. Also, as there was no longer a need to classify the new data by question type, we extracted the data automatically. This automation allowed us to include a more significant number of questions.
Finally, to finish the processing, since we are working with unstructured data, we used natural language processing techniques in the third activity to improve LDA results [56]. These included removing code snippets, non-ASCII characters, punctuation, and words with less than three characters. Afterward, we chose the set of stop words from [65] due to its size and availability on the Internet. Finally, we did not use word stemming algorithms because, in our empirical analysis, we realized that this process increases the difficulty of interpreting the topics.
Data Mining
For data mining, we performed topic modeling with the Latent Dirichlet Allocation algorithm. To do this, we used an R version of the Mallet tool [66]. As the number of topics depends on the problem investigated, and it is hard to define an idea number, we created models with different numbers of topics to evaluate the most suitable one empirically. We configured the tool to optimize hyperparameters every ten (10) iterations and train with 500 iterations. In the end, the chosen model had 15 topics for each eHealth subarea. The researchers hardly defined this number after a manual analysis of the models generated by the LDA, taking into account the trade-off between the model complexity and its representativeness.
On the other hand, for the study extension, we decided to use the n-gram data analysis [56] to identify the frequency of a contiguous sequence of n items from our mHealth question texts. The code programming of this technique was performed using Python libraries, and the link to the notebook is available in our supplementary information.
At the end of this phase, we have built some graphics using the Tableau software7 and the LDAvis8 method to assist the results evaluation process.
Evaluation
In the evaluation, all results were analyzed by two researchers. In addition, consolidating our open coding was carried out to define meaningful expressions that could characterize each discussion, highlighting recurring patterns.
Regarding LDA, initially, each researcher analyzed the topics independently using the LDAvis tool [54]. Then, the divergences were discussed in meetings. Finally, after interpreting the initial 45 topics (15 topics for each eHealth subarea), we decided to group some of them, considering their semantic similarity. For example, Google Fit, HealthKit, Core Motion, and ResearchKit were grouped into mHealth Frameworks. Finally, we got 19 meaningful topics in the first tree level and 45 specific topics.
Concerning the study extension, the n-gram frequency and some relevant questions based on the number of views, comments and answers were used to answer Research Question 4. We also promote a discussion correlating the data acquired in a previous systematic literature review related to mobile health [27], which was conducted by our research group.
Results and Discussion
Before discussing the Research Questions, it is crucial to understand the path of the initial investigation [28] and this expansion. In Fig. 6, we use the PRISMA flow chart structure to illustrate these paths.
We started our analysis by recovering 6082 eHealth questions in December 2019. After removing duplicates, we got 5608. Then, applying filters related to popularity and the discussion concept, we obtained 1076 posts. These posts made it possible to create 45 topics later refined to 19 meaningful topics.
At the beginning of April 2022, we recovered 1,832 mHealth questions. Then, sixteen (16) duplicates were removed, resulting in 1,816 questions. As previously discussed, we decided to focus on mobile health data in the study expansion. Thus, we did not apply the popularity and discussion filters. In addition, the data extraction was performed automatically.
Fig. 6 [Images not available. See PDF.]
Our results represented using the PRISMA flow chart
Regarding the temporal data distribution, we observed an increasing number of questions, especially in 2015 and 2016 (Fig. 7A).
Fig. 7 [Images not available. See PDF.]
A The number of eHealth questions over the years. B The distribution of eHealth questions considering the subareas: DICOM, EHR, and mHealth. In this graphic, we considered the complete set with all 5608 questions to get an overview of the trends without the popularity and discussion filters
To better understand this behavior, Fig. 7B presents this data separated by the subareas: DICOM, EHR, and mHealth. We noted that the number of questions about DICOM and EHR presents a linear growth based on these graphics. In contrast, between 2013 and 2015, the number of questions related to mHealth grew sharply. This growth is probably associated with the launching of two frameworks to support mobile applications’ development focused on healthcare: HealthKit (launched in September 2014) for the iOS platform and Google Fit (launched in October 2014) for the Android platform.
RQ1: What Technologies Have Been Discussed in eHealth?
Regarding RQ1, we performed a manual data extraction to characterize the technologies discussed in this area. We used an extraction form containing many fields, but not all fields presented significant data, probably because they are discussed in a transversal way. Thus, we discuss in this subsection the data for programming languages, operating systems, frameworks, APIs, libraries, and platforms.
As regard programming languages, the three most used languages for DICOM are Python [67], C# [68], and Matlab [69]. But, many questions address Java [70] and C++ [71] too. For EHR, the most discussed languages are Java, C#, and Javascript [72]. For mHealth, Swift, Java, and Objective-C [73] stand out. We observed that language usage is directly related to the development tools available in each subarea, as is the case of EHR with the HAPI Java API. Also, Python is widely used for image processing [74]. Swift and Objective-C are the basis for iOS development and Java for Android.
Concerning operating systems and frameworks data, we found a few posts (27) in the DICOM and EHR subareas from which it was possible to extract these data. Hence, it was not possible to draw conclusions linking these technologies and subareas. However, for the mHealth subarea, we found many discussions bringing information about the used operating system and framework.
In the case of mHealth discussions considering operating systems (OS) and frameworks, we found 213 of 380 (56.05%) questions about Health Kit and 110 (28.95%) related to Google Fit. Together these frameworks represent 85% of mHealth questions. Both frameworks seek to facilitate the management of health and fitness data for users of smartphones and wearables. The ResearchKit framework, despite the low number of related questions, deserves attention because it is an open-source framework created to support medical research. However, we also realized that the framework is directly related to the operating system. Thus, we found a more significant number of questions about iOS, followed by Android OS.
Finally, Table 2 summarizes the eHealth APIs, platforms, and libraries discussed in Stack Overflow. The sum of questions in this table is lower than the 1076 used in the analysis because there are questions in which we could not extract all the data. Besides, we suppressed technologies with just one question to improve the table visualization. However, our complete dataset is available online. Depending on the project’s context, this information can help practitioners choose the most suitable artifact.
Table 2. APIs, Platforms, and Libraries discussed in eHealth subareas
Subareas | Name | Description | # of questions |
|---|---|---|---|
APIs | |||
EHR | HAPI | HAPI is an open-source HL7 parser for Java | 23 |
nHAPI | It was developed to help.Net developers to manage HL7 messages | 5 | |
FHIR.Net API | It is used by developers to build FHIR client/server applications | 2 | |
mHealth | Fitbit API | API that enables communication with Fitbit devices like Fitbit Ionic | 7 |
Withings API | This API allows the development of apps using the Withings devices | 4 | |
Strava API | It enables the development of apps that use the Strava athletes data | 2 | |
Google Fit REST API | This API helps developers to manage health data from Google Fitness Store | 2 | |
Platforms | |||
DICOM | .Net | .Net is a Windows platform used to build different types of applications | 11 |
ClearCanvas | Platform for viewing, managing, and distributing DICOM images | 10 | |
NiftyNet | Open source CNN platform based on TensorFlow for medical image analysis | 6 | |
EHR | Mirth Connect | It is used to connect healthcare systems using standard like HL7 and FHIR | 35 |
.Net | .Net is a Windows platform used to build different applications | 10 | |
Node.js | Node.js supports the development of server-side JS applications | 6 | |
mHealth | Node.js | Node.js supports the development of server-side JS applications | 5 |
Xamarin | Xamarin is a.Net platform for developing native mobile apps | 3 | |
Ionic | Platform for web developers who want to build cross-platform mobile apps | 3 | |
.Net | .Net is a Windows platform used to build different applications | 3 | |
Libraries | |||
DICOM | pydicom | Python library to work with medical image datasets | 22 |
dcm4che | Set of libraries used in DICOM healthcare applications | 12 | |
Fellow Oak DICOM | DICOM library in C# for.Net Platform | 12 | |
Simple ITK | Set of tools for image analysis and was originally developed in C++ | 5 | |
GDCM | Grassroots DICOM is an open-source library to work with DICOM standard | 3 | |
JAI ImageIO | Java library that provides image processing and image analysis methods | 3 | |
pynetdicom | Python implementation of the DICOM networking protocol | 2 | |
Papaya | Javascript medical image viewer | 2 | |
RQ2: What eHealth Subjects Have Been Discussed in Stack Overflow?
For RQ2, we used the LDA to get the hot topics related to eHealth discussions. As a result, we obtained a multidimensional model that correlates words and documents. Usually, the evaluation of this type of model is performed by specialists using the terms present in the topics. However, this activity is error-prone [54]. Thus, many authors have proposed visualization methods to facilitate the evaluation of LDA models. Here, we used LDAvis [54].
We identified 45 topics and refined them to 19 more meaningful topics. Figure 8 represents these critical topics by the first tree level. Next, we use the second level to give more details about the subjects. Together, these topics provide a high-level view of the subjects discussed by eHealth developers.
Fig. 8 [Images not available. See PDF.]
Most discussed topics in Stack Overflow by eHealth developers
Regarding the DICOM subarea, most discussions deal with aspects such as Image Manipulation (34.81%) and Data Handling (22.24%). However, there is still a significant percentage of discussions about libraries (13.79%) and the DICOM Standard [75] (13.77%), more specifically about Data Representation and the use of UID tags. Python language is so prevalent in this subarea that we found a specific topic of questions about its application in medical image processing. We also identified a group of questions about extracting health information from images, questions focused on communication between DICOM servers, and discussions about the libraries used in this subarea.
Regarding the EHR, the most discussed subjects are HL7 (28.78%), Data formatting, storage and encoding (16.60%), and Communication issues (16.55%). We also have topics about Medical information (13.80%), FHIR (12.63%), Interoperability with Mirth Connect (6.70%), and OpenEHR (4.94%). As expected, we found many discussions about the HL7 and FHIR standards. These standards’ relevance is due to their adoption by healthcare companies and governments of many countries [76]. FHIR emerged as an evolution for HL7v3, but due to difficulties migrating to newer versions, many developers still use HL7v2 and HL7v3. Another important subject related to the standardization of electronic health records is OpenEHR. It provides open-source specifications for EHR systems [77]. This model has been strengthening as consumers, and healthcare professionals understand the benefits of a free and safe health data exchange.
For mHealth, the most discussed subjects are Framework (39.56%), Data Monitoring (26.81%), Development Issues (19.77%), Wearables (10.81%), and Security and Privacy Policies (3.05%). In this subarea, the step counting subject calls our attention due to many questions. This behavior was unexpected once many commercial applications already have this functionality because we expected this topic to be consolidated knowledge in this community. We raise two hypotheses related to this concern: (i) different mobile hardware generates different health measures, and commercial apps address these specificities internally; and (ii) there is a lack of approaches to ensure the correctness of measurements obtained by these apps.
Another subject that deserves attention is the difficulties faced when the developer needs to perform tasks in the background. Many developers still have problems using this feature, indicating a need for improvements in mHealth frameworks. We also observed a significant interest in monitoring health data such as heart rate and sleep using wearables. In fact, various studies have used wearables to detect atypical health conditions [6]. With the gradual transfer of this knowledge to the industry, discussions about wearables should increase.
Finally, we expect that the number of user interface discussions (4.5%) should increase in the following years from the understanding that the user experience is a crucial factor in the acceptance of these new healthcare technologies [78].
RQ3: What Types of Questions Asked by Developers Related to eHealth are Most Recurrent on Stack Overflow?
Due to the Stack Overflow community’s strengthening, the answers found on this website can be seen as an extension to the formal documentation for many technologies [14]. However, an essential step in understanding the dynamics of a particular area is to observe what type of questions developers ask [15]. In our case, we chose the taxonomy proposed by [14].
Fig. 9 [Images not available. See PDF.]
A discussions by the question types; and B questions with accepted and no accepted answer
Thus, the first step to answering RQ3 was focused on the type of question that eHealth developers ask. Figure 9A presents the types of questions considering each of the subareas. As expected, in all subareas, the how-to type presented the most number of questions. In fact, many developers use SO to find instructions about how to do a specific task, e.g., “How can I connect to the FitBit Zip over Bluetooth 4.0 LE?”.
After the how-to type, the distribution of the other types for each subarea was different. For the DICOM subarea, the second question type with the highest number of questions was novice. We observed during the manual analysis that a large number of questions contain passages such as “I’m new to processing DICOM images”. This information can indicate a substantial difficulty in starting with medical image processing due to the complexity of the technologies and standards present in this area. The third most significant question type was decision help. Many DICOM questions seek opinions on the use of technologies or the best way to architect a solution (see Question ID: 10390121). We found similar behavior in the EHR area, differing only in the order of the decision help and novice types.
For EHR, we also highlight a large number of questions of the conceptual type, e.g., “What Difference HL7 V3 and CDA?”. For the mHealth subarea, we found 64 questions classified as error and 46 classified as a review. The questions are more practical for this subarea, focusing on errors or requesting assistance with code review. For instance, we identified questions such as “Why am I getting the error with the Ionic plugin healthkit?”.
Figure 9B shows the number of questions with accepted and no accepted answers considering each subarea and the global set. The hypothesis related to this information is that the higher the percentage of questions with an accepted answer, the more mature the community is about the technologies being discussed. In this regard, DICOM has the most significant number of questions with accepted answers (64.39%), followed by EHR (56.52%) and mHealth (52.63%). mHealth is an area that has been gaining prominence in the last five years, and many aspects of the development of this type of solution still have not a community consensus.
Another finding of the RQ3 refers to the low number of questions focused on non-functional requirements (only 11 questions - 5 related to security and 6 related to performance). This fact is worrying due to the criticality of healthcare systems. These systems are expected to develop non-functional requirements such as security and performance.
After analyzing which type of discussions the eHealth developers are asking for, we decided to deepen the investigation by conducting a pair-reviewed open coding process. This process aims to extract and systematize knowledge from textual data. In DICOM, the main concerns are using unique identifiers (UID) of the DICOM standard, data for testing and validating applications, meaning and purpose of DICOM tags, manipulation of 3D images, and visualization of medical images in web browsers. As an example of questions related to these concerns, we have: “Is it true that DICOM Media Storage SOP Instance UID = SOP Instance UID?” (which has a score equals to 7 and 2,358 views); “Where is possible download.dcm files for free?” (with 30 as score and 58,540 views); “How to decide if a DICOM series is a 3D volume or a series of images?” (which has 9 as score and 3,767 views).
For EHR, the two main concerns are about the HL7 standard. First, there is a concern about data for testing and validating applications, similar to the DICOM. Second, interoperability and the different versions of the HL7 standard still generate discussions. An interesting question to show these concerns is “What Difference HL7 V3 and CDA?”.
Regarding mHealth, the main concerns are HealthKit usage, heart rate monitoring, security issues, and best practices to recover health data. In addition, we highlight the concern of dealing with data acquisition in the background due to its criticality for this application, its complexity related to battery consumption, and the recurrence of problems reported by developers. For example, the following question had more than 12,180 views and still has no accepted answer: “Healthkit background delivery when app is not running”. Another example of discussion close related to the main concerns in mHealth is “How is it possible that Google Fit app measures number of steps all the time without draining battery?”, which was visualized more than 74k times. By the way, this was the question with the most views in our dataset.
RQ4: What are the Challenges and Future Visions for Mobile Health from Developers’ Perspective?
In the previous three Research Questions, we discussed the technologies, the subjects, and the types of questions most frequently in the Stack Overflow eHealth discussions. This discussion was suitable for comprehending the prevalent dynamics of eHealth developers. Furthermore, we expanded the study with a new Research Question focused on mobile health challenges to complement this knowledge. Thus, aiming to answer RQ4, we have updated the data acquired in the initial study [28], performed an analysis of the most relevant questions considering the metrics score, views, answers, comments, and favorites, and analyzed the most frequent n-grams in this new dataset.
Table 3. Most relevant mHealth questions based on score, views, answer count, comment count, and favorite count metrics (ordered by score)
Question | Score | Views | Answers | Comments | Favorites |
|---|---|---|---|---|---|
iOS Use of HKObserverQuery background update completionHandler | 67 | 1740 | 1 | 1 | 6 |
How is it possible that Google Fit app measures number of steps all the time without draining battery? | 49 | 84098 | 7 | 1 | 11 |
Problem with access token in Strava API v3 GET all athlete activities | 27 | 14736 | 3 | 1 | 7 |
Step count retrieved through Google Fit API does not match Step count displayed in Google Fit Official App | 15 | 10490 | 4 | 2 | 11 |
HealthKit (iOS) won’t deliver data in background (objC) | 14 | 3969 | 2 | 17 | 8 |
Pedometer (Step Counter) | 13 | 36636 | 3 | 0 | 2 |
Get total step count for every date in HealthKit | 10 | 12276 | 6 | 0 | 7 |
Google Oauth 2.0 RESULT CANCELED while using Google Fit API | 9 | 10458 | 8 | 6 | 1 |
How to integrate Samsung Gear Steps in android Application? | 8 | 4153 | 2 | 19 | 2 |
How to call the function from other node js file | 5 | 33908 | 4 | 3 | 5 |
Flutter - Triggering Stream Subscription on Data Manually | 0 | 850 | 1 | 16 | 0 |
Table 3 presents the most relevant mobile health questions based on score, views, answers, views, and favorites. The complete dataset can be found in Supplementary Information. The questions in this table reinforce some of the observations identified earlier, for example:
Discussions are often related to two mHealth frameworks: Google Fit (for the Android operating system) and HealthKit (for the iOS operating system). Although there are issues linked to cross-platform frameworks like Flutter,9 native applications are predominant. This fact is probably related to the ease of using sensors in native apps. So, it is possible to highlight that there is still a gap regarding using cross-platform frameworks for healthcare.
Developers face many difficulties in performing background sensor data collection. A priori, this should not be a problem, as frameworks are expected to assist in this activity. However, as operating systems have strategies to avoid excessive resource consumption, background monitoring is a complex barrier to overcome. This problem is compounded when there is concern about battery consumption, as in the discussion How is it possible that Google Fit app measures number of steps all the time without draining battery?
Another frequent challenge refers to the sensor’s accuracy. For example, many developers report a discrepancy between the actual data and the data recorded by the mHealth frameworks. This divergence shows that there is still much room for improvement in identifying movements from accelerometer data, for instance. In fact, in the discussion of the question Step count retrieved through Google Fit API does not match Step count displayed in Google Fit Official App, a Google Fit software engineer came into the discussion to explain that the Google Fit app can be more accurate than its framework because they employ many heuristics and machine learning to improve their results.
Finally, we can observe that the maturity level of mHealth frameworks and the developer community should be improved since, among the most viewed questions, there are issues related to relatively simple apps such as step counters. This kind of application would be an initial case study incorporated into the framework’s documentation. Thus, even developers with little experience could easily overcome issues related to step counters.
Table 4. Ngram and their count
Ngram | Size | # | Ngram | Size | # |
|---|---|---|---|---|---|
(google, fit) | 2 | 1987 | (http, developer, google, com, fit) | 5 | 73 |
(fit, api) | 2 | 463 | (www, googleapis, com, fitness, user) | 5 | 46 |
(step, count) | 2 | 393 | (http, www, googleapis, com, fitness) | 5 | 46 |
(heart, rate) | 2 | 387 | (developer, google, com, fit, android) | 5 | 39 |
(fit, app) | 2 | 223 | (com, google, step, count, delta) | 5 | 35 |
(com, google) | 2 | 203 | (www, googleapis, com, auth, fitness) | 5 | 30 |
(using, google) | 2 | 189 | (http, www, googleapis, com, auth) | 5 | 30 |
(rest, api) | 2 | 178 | (com, fitness, user, dataset, aggregate) | 5 | 18 |
(data, google) | 2 | 176 | (google, step, count, delta, com) | 5 | 18 |
(access, token) | 2 | 168 | (googleapis, com, fitness, user, dataset) | 5 | 18 |
(google, fit, api) | 3 | 433 | (http, www, googleapis, com, fitness, user) | 6 | 46 |
(google, fit, app) | 3 | 189 | (http, developer, google, com, fit, android) | 6 | 39 |
(data, google, fit) | 3 | 158 | (http, www, googleapis, com, auth, fitness) | 6 | 30 |
(using, google, fit) | 3 | 156 | (googleapis, com, fitness, user, dataset, aggregate) | 6 | 18 |
(com, google, android) | 3 | 98 | (com, google, step, count, delta, com) | 6 | 18 |
(developer, google, com) | 3 | 97 | (www, googleapis, com, fitness, user, dataset) | 6 | 18 |
(http, developer, google) | 3 | 95 | (www, googleapis, com, fitness, user, datasources) | 6 | 16 |
(google, fit, rest) | 3 | 85 | (google, step, count, delta, com, google) | 6 | 16 |
(google, fit, data) | 3 | 85 | (step, count, delta, com, google, android) | 6 | 15 |
(fit, rest, api) | 3 | 85 | (derived, com, google, step, count, delta) | 6 | 15 |
(http, developer, google, com) | 4 | 95 | (http, www, googleapis, com, fitness, user, dataset) | 7 | 18 |
(google, fit, rest, api) | 4 | 80 | (www, googleapis, com, fitness, user, dataset, aggregate) | 7 | 18 |
(http, www, googleapis, com) | 4 | 76 | (http, www, googleapis, com, fitness, user, datasources) | 7 | 16 |
(developer, google, com, fit) | 4 | 73 | (com, google, step, count, delta, com, google) | 7 | 16 |
(using, google, fit, api) | 4 | 71 | (derived, com, google, step, count, delta, com) | 7 | 15 |
(googleapis, com, fitness, user) | 4 | 47 | (google, step, count, delta, com, google, android) | 7 | 15 |
(www, googleapis, com, fitness) | 4 | 46 | (developer, google, com, android, reference, com, google) | 7 | 14 |
(com, google, step, count) | 4 | 44 | (http, developer, google, com, android, reference, com) | 7 | 14 |
(google, com, fit, android) | 4 | 39 | (google, com, android, reference, com, google, android) | 7 | 14 |
(google, step, count, delta) | 4 | 35 | (com, android, reference, com, google, android, fitness) | 7 | 13 |
The results from the n-grams (Table 4) reinforce the observations of the most relevant questions. Thus, we can answer the first part of RQ4 stating that the primary mobile health challenges from the developers’ perspective are: background data collection, the accuracy of inferences made from sensor data, issues concerning data collection using Restful API, and initial documentation for the development of simple applications in healthcare.
Aiming to answer the second part of the RQ4, we decided to improve the mobile health analytical model proposed by [79]. This paper discusses the relationship between the mobile health area, technologies, applications, patients, and healthcare professionals. Our adapted model makes the relationship between these entities more explicit, includes the Health Intervention entity, and exemplifies some entities such as Application.
As stated by [79], Mobile health is more than just some healthcare applications on a mobile phone. According to Fig. 10, mobile health makes many possible kinds of applications such as emergency or loneliness monitoring, chronic care, fall detection, Quality of Life monitoring, and gait analysis. These applications assist patients (that means users) during their treatment. The patient can be a child, adult, or older. They can also have chronic or acute illnesses, and this health issue will make them dependent or independent. Naturally, the user can provide feedback on the mobile health area. Furthermore, healthcare professionals follow up with the patient, influence mobile health, and define health interventions (e.g., habit change and medical treatment). Finally, technologies like the Internet of Things and Machine Learning support mobile health.
Fig. 10 [Images not available. See PDF.]
Mobile Health analytical model. Adapted from [79]
In a previous work of our research lab [27], we have conducted a systematic mapping to identify state of the art in the literature concerned with the development of mobile applications for elderly healthcare, considering healthcare and software engineering viewpoints. In this mapping, we discussed four major future perspectives for mobile health: quality assurance, data privacy and security, data mining and machine learning, and user acceptance. We propose these perspectives following a literature point of view.
Now, it is possible to reinforce the perspectives acquired by the previous systematic mapping, including the developers’ point of view. Based on the results of this paper, we can conclude that in the next few years, we should observe a growing interest (from both researchers and healthcare practitioners) in background monitoring that prioritizes optimized resource consumption, improves the accuracy of Machine Learning techniques used to infer activities daily living, and makes health APIs easier. The convergence between both points of view is the need for new methods for user engagement and the adoption of Machine Learning.
Findings Summary and Practical Implications
This work analyzes a decade of the most popular eHealth discussion in the Stack Overflow. Our results can be used to understand the trends and difficulties faced by eHealth developers. It can also be helpful for practitioners who want to know which technologies are most used in eHealth. In this subsection, we summarize our findings and present their practical implications.
Concerning technologies (RQ1), each subarea has adopted more tools and, consequently, more discussions on Stack Overflow. For example, DICOM has a library written in Python called pydicom. On the other hand, EHR has many questions about the open-source Java API called HAPI. Moreover, for mHealth, two frameworks stand out: HealthKit for the iOS operating system using Swift and Objective-C and Google Fit for the Android operating system using Java.
We identify many DICOM questions about image manipulation and data handling related to subjects discussed in the eHealth area (RQ2). In EHR, the most frequent subjects are the HL7 standard, data formatting, storage and encoding, and communication issues. For mHealth, we have discussions about frameworks (mainly Google Fit and HealthKit), data monitoring, development issues, wearables, and security and privacy policies.
In RQ3, we found that how-to questions are prevalent in all three subareas considered in this study. DICOM and EHR have many questions classified as decision help and novice. This information can indicate a more significant barrier for developers starting in these subareas. mHealth, differently, has many questions that present specific errors or that request code snippets review. Another point that deserves attention is the low number of non-functional questions, which may indicate low interest in non-functional requirements.
Deepening the RQ3 analysis, we found the following concerns: (i) interpretation and usage of the DICOM and HL7 standards; (ii) manipulation of 3D images; (iii) DICOM web viewer; (iv) data for testing for both DICOM and EHR applications; (v) interoperability using Mirth Connect; (vi) data types and background execution with HealthKit; (vii) heart rate monitoring; viii) authentication and permission, and (ix) best practices to query health data.
In RQ4, we found results that corroborate the previous research questions’ findings. The mobile health discussions on Stack Overflow focus on two frameworks: Google Fit and HealthKit. The main challenges developers face are monitoring health data in the background, communicating with the framework APIs, and the accuracy of inferences made from the healthcare data.
Finally, in addition to present a systematic analysis of the eHealth knowledge mined in Stack Overflow and discussing a snapshot of (more than) ten years of discussions in this area, this study also has other practical implications:
Research opportunities this study can also represent a kick-off for further investigations regarding the gaps found, such as surveying with eHealth developers to understand better the difficulties faced when working with mHealth; or propose testing approaches focused on health tracking apps. Also, we can raise the following research questions for further investigation:
What strategies can optimize computational resources, especially battery when monitoring users’ health data is needed?
Why does the eHealth developer community face so many troubles when communicating with the leading frameworks APIs? Is the problem in the documentation, in the developer’s maturity level, or both?
Which techniques can improve the accuracy of inferences made from healthcare data? For example, what techniques can improve step detection from data collected on smartphones?
Assuming that “your smartphone knows you better than you think” [80], what data can be used to train intelligent models capable of inferring the users’ Quality of Life level?
Healthcare is a critical area for security, and privacy [81] even though there are few discussions on this subject within the eHealth developer community. What is missing to make these discussions even more frequent?
Considering the difficulties in validating healthcare applications and the restrictions on obtaining data that can be used in testing, how can the runtime testing technique [82] be applied to healthcare to ensure the privacy of data used in testing?
Beginners guide: for practitioners starting to work with eHealth, this paper can support the decision about which tool to use. We have seen a strengthening of Python tools for medical image processing, the consolidation of Java tools for EHR, and polarization between HealthKit and Google Fit for mobile health.
Challenges and Limitations
Discussing limitations and threats is essential when conducting evidence-based software engineering studies [13]. It helps us to understand the outcome of confidence. Our study identified some threats to validity and sought to mitigate them through the conducting phase.
The first threat showed up while selecting the data source and target tags. Despite the choice of just one database, the English version of Stack Overflow has high representativeness of developers’ discussions. This website has also been used in many scientific studies [15, 17].
We sought to follow a systemic approach to reduce bias in choosing the tags to compose our query. Initially, we considered eHealth synonyms defined by the MeSH vocabulary as tags. This controlled vocabulary has significant relevance for indexing papers in the life sciences, and it is used as a reference to support the building of search strings for systematic reviews [83, 84]. However, we faced a limitation concerning the different terminology used by researchers and practitioners. When we tried to use the terms suggested by MeSH, no results were found. With that, we decided to use a snowballing strategy to include all health-related tags. This strategy returned a significant number of questions. We understand that this decision can raise another threat once some questions could be left out of the study. However, after a detailed analysis of the questions, we considered them suitable to draw paper conclusions. Also, the manual analysis removed noises, i.e., questions whose tag does not match the content of the question. In this step, we found and removed 36 discussions classified as noise.
Regarding data volume, the filters applied in this study followed criteria already validated in the literature [49, 63]. We chose to analyze only the most relevant questions for the eHealth community using the distribution of score metrics. Although small, our final dataset (1,076 questions) is highly representative. Further investigations can use the complete dataset.
We also sought to mitigate manual data extraction bias threats involving two researchers in this activity. We held meetings to discuss the results and the topic interpretation. In these meetings, we noticed some limitations. For example, the technologies identified in the Stack Overflow questions were classified in API, frameworks, libraries, or platforms, considering their description. However, we noted that these descriptions do not always follow definitions already established in software engineering. Thus, adopting clear definitions for the software artifacts in future work would be interesting.
Another limitation of our analysis concerns the developers’ profiles. In practice, there are several types of developers (such as mobile, web, and full-stack developers), and we did not distinguish between these profiles. Nevertheless, considering these profiles, this point can represent an exciting opportunity to compare our results.
Regarding the topic interpretation, we sought to improve its reliability by conducting—with two researchers—an evaluation using different topic numbers. This evaluation considered the trade-off between model complexity and the topic’s representativeness, and it was performed using the LDAvis.
Related Work
Several papers use data mining to analyze patterns in software repositories. There are also many studies seeking to map eHealth papers systematically. However, we did not find papers focused on observing this area’s behavior from ICT professionals’ perspectives and using Stack Overflow as a data source. Although we found one paper that uses Stack Overflow for data mining related to health applications, this paper focuses on the development of solutions related COVID-19 pandemic. Thus, most papers in this section are related to our work by the research method or the study area.
In work [85], the authors present the results of a data mining performed using the Stack Overflow question and answer site with a focus on relevant issues related to the development of software systems related to the COVID-19 pandemic and their interrelationships. This paper analyzed 2719 posts with COVID-19-related questions on Stack Overflow through October 2020. The research focused on overviewing COVID-19-related discussions on Stack Overflow and identifying research advances and issues facing COVID-19 associated with software development. Descriptive Statistics, Association Rule Graphs (ARG), Survival Analysis (SA), and Latent Dirichlet Allocation (LDA) were employed for the post-analysis. The results show that the response of the developers’ community to the pandemic was immediate and that the interest of developers in COVID-19-related challenges remained continuous, even after an initial peak.
Furthermore, the results showed that the main questions were related to technological problems, Web Frameworks, and Big Data/ML. In contrast, questions about Platforms, Databases, Developer Tools, and Collaboration Tools accumulate a few question posts, which indicates the lower popularity of these specific technological aspects in COVID-19 software development. Also, according to the authors, regarding the problems addressed, the results focused on COVID-19 data collection, analysis, and visualization from/to the web, in line with the general needs for monitoring the pandemic.
The approach used and the focus of our research differ from that presented in this work. In particular, our research objective was to identify the main topics the Stack Overflow developer community addressed about the development of health applications and not just the topics related to the COVID-19 pandemic. However, it is important to note how much the COVID-19 pandemic impacted the increase in the search for health software solutions.
The work developed in [53] has an objective similar to that proposed in our work. The authors modeled topics based on the PubMed Digital Library papers to extract trends in this literature. They used the MeSH-controlled vocabulary to build the search string. They recovered 25,824 publications until December 2016. After the refinement stage, they got 19,825 papers. The LDA was applied to the titles, keywords, and abstracts, considering 160 topics experts later reviewed. The most frequent topics were wearables, randomized control trials, legal issues & ethics, eye disease, and teleconsultation. The difference in our approach lies in characterizing the area from the professionals’ perspective.
[86] used probabilistic techniques to analyze the literature related to the eHealth field. To compose the study corpus, the authors considered the titles and abstracts of 23,988 articles (collected in PubMed Digital Library between December 31, 2017, and May 8, 2018). The topic modeling identified 100 meaningful subjects in the service model, disease, behavior, and lifestyle categories. The results indicated a shift in focus from the DICOM to the mHealth subarea. We also observed an increase in Stack Overflow discussions focused on mobile health development, reinforcing this trend.
Already in [87] is present a systematic review of reviews (i.e., a tertiary study) carried out to identify research opportunities in this area. The authors analyzed 47 papers published in several digital libraries between January 2010 and June 2017. As a result, they highlight five challenging areas (stakeholders and system users, technology and interoperability, cost-effectiveness and start-up costs, legal clarity and legal framework, and local context and regional differences) and four areas of opportunity (participation and contribution, foundation and sustainability, improvements and productivity, and identification and application).
In [88], a literature review on wearable IoT applications was prepared, where 44 works were evaluated, among which 8 Surveys and 36 articles were published between 2017 and 2022. Based on these works, IoT applications were identified for several areas with an emphasis on medical applications aimed at health care. In particular, researchers found a set of still-unanswered questions related to non-functional security, integrity, and interoperability requirements. In addition, the devices’ low processing power and energy limitations stand out as challenges to developing healthcare apps for wearable devices. It also highlighted the discomfort for patients when using wearables as one of the main difficulties for greater adherence to mobile healthcare applications for IoT wearable devices.
Lastly, in [89], the researchers developed a web-based survey aimed at mobile health app developers to identify the main issues to be considered for developing secure mobile health care applications. A total of 97 responses were obtained, allowing us to identify sets of security challenges, among which we highlight: the insufficient security knowledge of most mobile health app developers, the lack of budget reserve for the development of security measures in the apps, the typical lack of security experts in development teams, and lack of security testing during the development of these applications. The study also identified some essential practices and motivating factors to be considered during the development of MHealth applications.
Regarding the stakeholder and system users’ challenges, the authors mentioned the need to better integrate functional and non-functional requirements, mainly security, into the design and implementation of eHealth applications. Our results corroborate this point of view since there are few discussions about non-functional requirements for eHealth within Stack Overflow. Concerning the technology and interoperability challenges, the authors noted that despite standards such as FHIR, its adoption is still slow and requires much effort. Our data shows many questions in which the developers are looking for support to make decisions about EHR standards. In addition, many discussions report difficulties with legacy systems that use older versions of HL7. Finally, unlike the other papers, our work performed a topic modeling with LDA to find trends and demands in Stack Overflow eHealth questions, taking into account the perspective of ICT professionals. Table 5 compares our proposal and related works.
Table 5. Comparison between our proposal and the related works
Work | Method/technique | Data source | Perspective | Results |
|---|---|---|---|---|
[85] | Statistics with ARG, SA, and LDA | Stack Overflow | ICT Professionals | COVID-19 trends |
[53] | Topic Modeling with LDA | PubMed Papers | Researchers | Literature trends |
[86] | Probabilistic techniques | PubMed Papers | Researchers | Literature trends |
[87] | Systematic Review of Reviews | Review Papers | Researchers | Challenges |
[88] | Literature Review | Surveys and Papers | Researchers | IoT Wearable Challenges |
[89] | Survey with Developers | Web Based Survey | Developers | Security Challenges |
Our work | Topic Modeling with LDA | Stack Overflow Questions | ICT Professionals | Trends and Demands |
Final Remarks
The eHealth term was defined almost 20 years ago. Since then, this area has faced many changes driven by the emergence of new healthcare technologies. Recently, with the cost reduction of wearable devices and due to its ability to monitor many different aspects of its users’ health, this area has gained a new momentum with mobile health. Thus, it is possible to find many papers proposing new technological artifacts and other studies that seek to map these advances systematically.
On the other hand, this work proposed an investigation into the discussions associated with eHealth development, considering the practitioners’ perspectives and using the Q &A website Stack Overflow as the data source. Initially, we found 6,082 posts related to eHealth solutions in December 2019. Then, after removing duplicates and applying popularity filters, we select 1076 discussions. So, we used manual extraction and topic modeling to understand the behavior of eHealth developers in SO. In this first research, we have done a descriptive analysis to understand the eHealth area from the developers’ point of view. Then, at the beginning of April 2022, we expanded our research, and after we removed duplicates, we obtained 1816 questions related to mHealth from Stack Overflow. Finally, due to its relevance, we decided to focus on mobile health data in this study.
In our initial study [28], we observed a growing trend regarding mHealth discussions. The data also revealed three clusters of questions in SO: DICOM, EHR, and mHealth. The most frequent discussions in the DICOM (that includes questions about digital medical image processing) and EHR (with issues focusing on Electronic Health Records) clusters are related to the decision-making process during the development of solutions and the assistance to novices. In mHealth (which includes questions about mobile health applications), the discussions are more technical and specific, focusing on error resolution and code review. We also found a few issues associated with non-functional requirements despite the relevance of safety, performance, and usability for health applications. After expanding the study, we noted that the results reinforced our previous analysis of mHealth cluster topics.
Also, we identify a direct correlation between programming languages and the most used artifact in the subareas regarding technologies. Python and the pydicom library for DICOM, Java and the HAPI API for EHR, Swift, and the HealthKit framework for mHealth stand out. After interpreting topics, we concluded that image manipulation is the most discussed subject in DICOM. In EHR, it is the HL7 standard, and for mHealth is the framework for development. Finally, we found that the most significant concerns arising from conceptual issues are understanding DICOM and HL7 standards, data for testing and validating, and monitoring health data in the background.
The study expansion, which focused on mHealth data, showed that most mHealth discussions on Stack Overflow are related to two native development frameworks: Google Fit (for Android) and HealthKit (for iOS). Thus, it was evident that there is still a gap regarding cross-platform frameworks applied to healthcare, mainly due to difficulties in collecting sensor data. Furthermore, another observation of this new study refers to the main challenges that mHealth developers face. In general, there are many difficulties for monitoring in the background, especially when it is required to optimize computational resources (e.g., battery); getting data from frameworks APIs; inconsistencies regarding the inferences made from the health data (for example, there are many discrepancies between the actual number of steps and the number inferred from the accelerometer data).
This work can help practitioners know trends like the most used technologies in this area. Moreover, this work can help researchers identify opportunities for future work. Among these are the need to develop more suitable techniques for testing healthcare applications, the availability of datasets that assist these activities, and the investigation of why there are few questions about non-functional requirements in the eHealth area. Also, other research opportunities can involve improving DICOM and HL7 standards and investigating a tool to help developers deal with monitoring data in the background and optimize battery consumption of the mHealth applications.
Acknowledgements
The authors would like to thank CNPQ for the Productivity Scholarship of Rossana M. C. Andrade DT-1 ( 306362 / 2021-0), for the Productivity Scholarship of Pedro de A. dos Santos Neto DT-2 ( 315198/2018-4), and CAPES that provided to the Evilasio C. Junior a Ph.D. scholarship.
Availability of data and material
All data used are available on the internet to ensure the study reproducibility and future in-depth analysis. Queries: data.stackexchange.com/users/32389 First dataset: bit.ly/3cHPEPL Second dataset: bit.ly/3MmAX64LDA and N-gram notebook: github.com/pedroalmir/trends-ehealth Higher resolution images: bit.ly/3lhKhvP.
Declarations
Conflict of interest
The authors certify that they have NO affiliations with or involvement in any organization or entity with any financial or non-financial interest in the subject matter or materials discussed in this manuscript.
Research involving Human and/or Animals
Not applicable.
Informed consent
Not applicable.
We are social Report: https://datareportal.com/reports/digital-2022-global-overview-report.
2Stack Overflow website: https://stackoverflow.com/company.
3Stack Exchange website: http://data.stackexchange.com.
4Stack Overflow Survey: https://insights.stackoverflow.com/survey/2019.
5MeSH: ncbi.nlm.nih.gov/mesh.
6SO tag search system: https://stackoverflow.com/tags.
7Tableau website: https://www.tableau.com.
8LDAvis repository: https://github.com/cpsievert/LDAvis.
9Flutter website: https://flutter.dev/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. Eysenbach, G. What is e-health?. J Med Internet Res; 2001; 3,
2. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, McKinstry B, Procter R, Majeed A, Sheikh A. The impact of ehealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8(1).
3. Andrade R, Carvalho R, de Araújo I, Oliveira K, Maia M. What changes from ubiquitous computing to internet of things in interaction evaluation? In: International Conference on Distributed, Ambient, and Pervasive Interactions, 2017;3–21. Springer.
4. Masood, A; Sheng, B; Li, P; Hou, X; Wei, X; Qin, J; Feng, D. Computer-assisted decision support system in pulmonary cancer detection and stage classification on ct images. J Biomed Inform; 2018; 79, pp. 117-128. [DOI: https://dx.doi.org/10.1016/j.jbi.2018.01.005]
5. Peterson KJ, Jiang G, Liu H. A corpus-driven standardization framework for encoding clinical problems with hl7 fhir. Jo Biomed Inform. 2020;103541.
6. Almeida RL, Macedo AA, de Araújo ÍL, Aguilar PA, Andrade RM:.Watchalert: Uma evolução do aplicativo falert para detecção de quedas em smartwatches. In: Anais Estendidos do XXII Simpósio Brasileiro de Sistemas Multimídia e Web, 2016;124–127. SBC
7. de Araujo, FHD; Santana, AM; dos Santos, N. PdA: Evaluation of classifiers based on decision tree for learning medical claim process. IEEE Latin Am Transa; 2015; 13,
8. Araújo, FH; Santana, AM. Neto, PdAS: using machine learning to support healthcare professionals in making preauthorisation decisions. Int J Med Inform; 2016; 94, pp. 1-7. [DOI: https://dx.doi.org/10.1016/j.ijmedinf.2016.06.007]
9. Gaddi, A; Capello, F; Manca, M. eHealth.; 2013; London, Springer:
10. Ponzanelli L, Bacchelli A, Lanza M. Seahawk: Stack overflow in the ide. In: 35th Int. Conf. on Soft. Engineering (ICSE). 2013;1295–1298.
11. Silva R, Roy C, Rahman M, Schneider K, Paixao K, Maia M. Recommending comprehensive solutions for programming tasks by mining crowd knowledge. In: 2019 IEEE/ACM 27th Int. Conf. on Program Comprehension, 2019;358–368. IEEE
12. Barua, A; Thomas, SW; Hassan, AE. What are developers talking about? An analysis of topics and trends in stack overflow. Emp Softw Eng; 2014; 19,
13. Kitchenham BA, Budgen D, Brereton P. Evidence-based Software Engineering and Systematic Reviews vol. 4. CRC press 2015.
14. Treude C, Barzilay O, Storey M-A. How do programmers ask and answer questions on the web?: Nier track. In: 2011 33rd International Conference on Software Engineering (ICSE), 2011;804–807. IEEE.
15. Beyer S, Macho C, Di Penta M, Pinzger M. What kind of questions do developers ask on stack overflow? a comparison of automated approaches to classify posts into question categories. Emp Softw Eng. 2019;1–44.
16. Pérez-López R, Blanco G, Fdez-Riverola F, Lourenço A. The activity of bioinformatics developers and users in stack overflow. In: International Conference on Practical Applications of Computational Biology & Bioinformatics, 2020;23–31. Springer.
17. Chen, H; Coogle, J; Damevski, K. Modeling stack overflow tags and topics as a hierarchy of concepts. J Syst Softw; 2019; 156, pp. 283-299. [DOI: https://dx.doi.org/10.1016/j.jss.2019.07.033]
18. Ragkhitwetsagul C, Krinke J, Paixao M, Bianco G, Oliveto R. Toxic code snippets on stack overflow. IEEE Trans Softw Eng. 2019.
19. Wu, Y; Wang, S; Bezemer, C-P; Inoue, K. How do developers utilize source code from stack overflow?. Emp Softw Eng; 2019; 24,
20. Culotta A. Towards detecting influenza epidemics by analyzing twitter messages. In: Proceedings of the First Workshop on Social Media Analytics, 2010;115–122. ACM.
21. Paul, MJ; Dredze, M. Discovering health topics in social media using topic models. PloS One; 2014; 9,
22. Nguyen T, Nguyen DT, Larsen ME, O’Dea B, Yearwood J, Phung D, Venkatesh S, Christensen H. Prediction of population health indices from social media using kernel-based textual and temporal features. In: Proceedings of the 26th International Conference on World Wide Web Companion, 2017;99–107. International World Wide Web Conferences Steering Committee.
23. Kwon J, Grady C, Feliciano JT, Fodeh SJ. Defining facets of social distancing during the covid-19 pandemic: Twitter analysis. medRxiv 2020.
24. Chiarini, G; Ray, P; Akter, S; Masella, C; Ganz, A. mhealth technologies for chronic diseases and elders: a systematic review. IEEE J Select Areas Commun; 2013; 31,
25. Gagnon, M-P; Ngangue, P; Payne-Gagnon, J; Desmartis, M. m-health adoption by healthcare professionals: a systematic review. J Am Med Inform Assoc; 2015; 23,
26. Robbins, TD; Keung, SNLC; Arvanitis, TN. E-health for active ageing; a systematic review. Maturitas; 2018; 114, pp. 34-40. [DOI: https://dx.doi.org/10.1016/j.maturitas.2018.05.008]
27. Paiva, JO; Andrade, RM; de Oliveira, PAM; Duarte, P; Santos, IS. Evangelista, ALdP, Theophilo, RL, de Andrade, LOM, Barreto, ICdH: Mobile applications for elderly healthcare: A systematic mapping. PloS One; 2020; 15,
28. Oliveira P, Costa Junior E, Santos IDS, Andrade R, Santos Neto PdA. Ten years of ehealth discussions on stack overflow. In: International Conference on Health Informatics (HEALTHINF1’22) 2022.
29. Ullah M, Fiedler M, Wac K. On the ambiguity of quality of service and quality of experience requirements for ehealth services. In: 2012 6th International Symposium on Medical Information and Communication Technology (ISMICT), 2012;1–4. IEEE
30. Sahama T, Simpson L, Lane B. Security and privacy in ehealth: is it possible? In: 15th International Conference on e-Health Networking, Applications and Services (Healthcom), 2013;249–253. IEEE.
31. Farahani, B; Firouzi, F; Chang, V; Badaroglu, M; Constant, N; Mankodiya, K. Towards fog-driven iot ehealth: promises and challenges of iot in medicine and healthcare. Fut Gen Comput Syst; 2018; 78, pp. 659-676. [DOI: https://dx.doi.org/10.1016/j.future.2017.04.036]
32. Bostrom, J; Sweeney, G; Whiteson, J; Dodson, JA. Mobile health and cardiac rehabilitation in older adults. Clini Cardiol; 2020; 43,
33. Qudah, B; Luetsch, K. The influence of mobile health applications on patient-healthcare provider relationships: a systematic, narrative review. Patient Educ Counsel; 2019; 102,
34. Curran, K; Nichols, E; Xie, E; Harper, R. An intensive insulinotherapy mobile phone application built on artificial intelligence techniques. J Diabet Sci Technol; 2010; 4,
35. Ribu, L; Holmen, H; Torbjørnsen, A; Wahl, AK; Grøttland, A; Småstuen, MC; Elind, E; Bergmo, TS; Breivik, E; Årsand, E et al. Low-intensity self-management intervention for persons with type 2 diabetes using a mobile phone-based diabetes diary, with and without health counseling and motivational interviewing: protocol for a randomized controlled trial. JMIR Res Protocols; 2013; 2,
36. Toro-Ramos, T; Kim, Y; Wood, M; Rajda, J; Niejadlik, K; Honcz, JY; Marrero, D; Fawer, A; Michaelides, A. Efficacy of a mobile hypertension prevention delivery platform with human coaching. J Hum Hyperten.; 2017; 31,
37. Toro-Ramos, T; Lee, D-H; Kim, Y; Michaelides, A; Oh, TJ; Kim, KM; Jang, HC; Lim, S. Effectiveness of a smartphone application for the management of metabolic syndrome components focusing on weight loss: a preliminary study. Metab Synd Relat Disord; 2017; 15,
38. Portz, JD; Vehovec, A; Dolansky, MA; Levin, JB; Bull, S; Boxer, R. The development and acceptability of a mobile application for tracking symptoms of heart failure among older adults. Telemed e-Health; 2018; 24,
39. Elias P, Rajan NO, McArthur K, Dacso CC. Inspire to promote lung assessment in youth: evolving the self-management paradigms of young people with asthma. Medicine 2.0 2013;2(1).
40. Ribeiro, N; Moreira, L; Almeida, AMP; Santos-Silva, F. Pilot study of a smartphone-based intervention to promote cancer prevention behaviours. Int J Med Inform; 2017; 108, pp. 125-133. [DOI: https://dx.doi.org/10.1016/j.ijmedinf.2017.10.013]
41. Klasnja P, Consolvo S, McDonald DW, Landay JA, Pratt W. Using mobile and personal sensing technologies to support health behavior change in everyday life: lessons learned. In: AMIA Annual Symposium Proceedings, 2009;2009:338. American Medical Informatics Association.
42. Gabrielli, S; Dianti, M; Maimone, R; Betta, M; Filippi, L; Ghezzi, M; Forti, S et al. Design of a mobile app for nutrition education (trec-lifestyle) and formative evaluation with families of overweight children. JMIR mHealth and uHealth; 2017; 5,
43. Ming, LC; Untong, N; Aliudin, NA; Osili, N; Kifli, N; Tan, CS; Goh, KW; Ng, PW; Al-Worafi, YM; Lee, KS et al. Mobile health apps on covid-19 launched in the early days of the pandemic: content analysis and review. JMIR mHealth and uHealth; 2020; 8,
44. Organization WH, et al. Clinical management of severe acute respiratory infection (sari) when covid-19 disease is suspected: interim guidance, 13 March 2020. In: Technical report: World Health Organization; 2020.
45. Organization WH. et al. Classification of digital health interventions v1.0: a shared language to describe the uses of digital technology for health. Technical report, World Health Organization. 2018.
46. Hensher, M; Cooper, P; Dona, SWA; Angeles, MR; Nguyen, D; Heynsbergh, N; Chatterton, ML; Peeters, A. Scoping review: development and assessment of evaluation frameworks of mobile health apps for recommendations to consumers. J Am Med Inform Assoc; 2021; 28,
47. Nadal, C; Sas, C; Doherty, G et al. Technology acceptance in mobile health: scoping review of definitions, models, and measurement. J Med Internet Res; 2020; 22,
48. Mumtaz S, Rodriguez C, Benatallah B. Expert2vec: Experts representation in community question answering for question routing. In: International Conference on Advanced Information Systems Engineering, 2019;213–229. Springer.
49. Bandeira A, Medeiros CA, Paixao M, Maia PH. We need to talk about microservices: an analysis from the discussions on stackoverflow. In: 16th International Conference on Mining Software Repositories, 2019;255–259. IEEE Press
50. Mei Q, Cai D, Zhang D, Zhai C. Topic modeling with network regularization. In: Proceedings of the 17th International Conference on World Wide Web, 2008;101–110.
51. Blei, DM; Ng, AY; Jordan, MI. Latent dirichlet allocation. J Mach Learn Res; 2003; 3,
52. Sukhija N, Tatineni M, Brown N, Van Moer M, Rodriguez P, Callicott S. Topic modeling and visualization for big data in social sciences. In: 2016 Intl IEEE Conferences on Ubiquitous Intelligence and Computing, Advanced and Trusted Computing, Scalable Computing and Communications, Cloud and Big Data Computing, Internet of People, and Smart World Congress, 2016;1198–1205. IEEE.
53. Drosatos G, Kavvadias SE, Kaldoudi E. Topics and trends analysis in ehealth literature. In: EMBEC & NBC 2017, 2017;563–566. Springer, London.
54. Sievert C, Shirley K. Ldavis: A method for visualizing and interpreting topics. In: Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces, 2014;63–70.
55. Gardner MJ, Lutes J, Lund J, Hansen J, Walker D, Ringger E, Seppi K. The topic browser: An interactive tool for browsing topic models. In: Nips Workshop on Challenges of Data Visualization, 2010;2. Whistler Canada.
56. Thomas SW, Hassan AE, Blostein D. Mining unstructured software repositories. In: Evolving Software Systems. Springer, Berlin. 2014.
57. Bespalov D, Bai B, Qi Y, Shokoufandeh A. Sentiment classification based on supervised latent n-gram analysis. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, 2011;375–382.
58. Ahmed H, Traore I, Saad S. Detection of online fake news using n-gram analysis and machine learning techniques. In: International Conference on Intelligent, Secure, and Dependable Systems in Distributed and Cloud Environments, 2017;127–138. Springer.
59. Nessa S, Abedin M, Wong WE, Khan L, Qi Y. Software fault localization using n-gram analysis. In: International Conference on Wireless Algorithms, Systems, and Applications, 2008;548–559. Springer.
60. Fiordelli, M; Diviani, N; Schulz, PJ et al. Mapping mhealth research: a decade of evolution. J Med Internet Res; 2013; 15,
61. Fayyad, U; Piatetsky-Shapiro, G; Smyth, P. From data mining to knowledge discovery in databases. AI Mag; 1996; 17,
62. Wohlin C. Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: 18th Int. Conf. on Evaluation and Assessment in Software Engineering, 2014;1–10.
63. Kavaler D, Posnett D, Gibler C, Chen H, Devanbu P, Filkov V. Using and asking: Apis used in the android market and asked about in stackoverflow. In: International Conference on Social Informatics, 2013;405–418. Springer.
64. Stol K-J, Ralph P, Fitzgerald B. Grounded theory in software engineering research: a critical review and guidelines. In: Proceedings of the 38th Int. Conf. on Software Engineering, 2016;120–131.
65. Puurula A. Cumulative progress in language models for information retrieval. In: Proc. Australasian Language Technology Association Work. 2013 (ALTA 2013), Brisbane, Australia, 2013;96–100. www.aclweb.org/anthology/U13-1013.
66. McCallum AK. Mallet: A machine learning for language toolkit. 2002. http://mallet.cs.umass.edu.
67. Van Rossum, G; Drake, FL. The Python Language Reference Manual; 2011; London, Network Theory Ltd.:
68. Hejlsberg, A; Wiltamuth, S; Golde, P. The C# Programming Language; 2006; New Jersey, Adobe Press:
69. Herniter, ME. Programming in MATLAB; 2000; London, Brooks/Cole Publishing Co.:
70. Arnold, K; Gosling, J; Holmes, D; Holmes, D. The Java Programming Language; 2000; Reading, London, Addison-Wesley:
71. Stroustrup B. The c++ programming language: reference manual. Bell Lab: Technical report; 1984.
72. Flanagan D, Matilainen P. JavaScript. London: Anaya Multimedia; 2007.
73. Knaster, S; Dalrymple, M. Learn Objective-C on the Mac; 2009; London, Springer:
74. Van der Walt, S; Schönberger, JL; Nunez-Iglesias, J; Boulogne, F; Warner, JD; Yager, N; Gouillart, E; Yu, T. scikit-image: image processing in python. PeerJ; 2014; 2, 453. [DOI: https://dx.doi.org/10.7717/peerj.453]
75. Mildenberger, P; Eichelberg, M; Martin, E. Introduction to the dicom standard. Eur Radiol; 2002; 12,
76. Bender D, Sartipi K. Hl7 fhir: an agile and restful approach to healthcare information exchange. In: Proceedings of the 26th IEEE Int. Symposium on Computer-based Medical Systems, 2013;326–331.
77. Kalra, D; Beale, T; Heard, S. The openehr foundation. Stud Health Tech Inform; 2005; 115, pp. 153-173.
78. Zapata, BC; Fernández-Alemán, JL; Idri, A; Toval, A. Empirical studies on usability of mhealth apps: a systematic literature review. J Med Syst; 2015; 39,
79. Varshney, U. Mobile health: four emerging themes of research. Decis Sup Syst; 2014; 66, pp. 20-35. [DOI: https://dx.doi.org/10.1016/j.dss.2014.06.001]
80. Ghosh S, Löchner J, Mitra B, De P. Your smartphone knows you better than you may think: Emotional assessment ‘on the go’via tapsense. In: Quantifying Quality of Life, 2022;209–267. Springer, London.
81. Appari, A; Johnson, ME. Information security and privacy in healthcare: current state of research. Int J Internet Enterp Manag; 2010; 6,
82. dos Santos, EB; Andrade, RM; de Sousa Santos, I. Runtime testing of context-aware variability in adaptive systems. Inform Softw Technol; 2021; 131, [DOI: https://dx.doi.org/10.1016/j.infsof.2020.106482]
83. Lynch, BM; Matthews, CE; Wijndaele, K. New mesh for sedentary behavior. J Phys Act Health; 2019; 16,
84. Salvador-Oliván, JA; Marco-Cuenca, G; Arquero-Avilés, R. Errors in search strategies used in systematic reviews and their effects on information retrieval. J Med Lib Assoc JMLA; 2019; 107,
85. Georgiou, K; Mittas, N; Chatzigeorgiou, A; Angelis, L. An empirical study of covid-19 related posts on stack overflow: topics and technologies. J Syst Softw; 2021; 182, [DOI: https://dx.doi.org/10.1016/j.jss.2021.111089]
86. Drosatos, G; Kaldoudi, E. A probabilistic semantic analysis of ehealth scientific literature. J Telemed Telecare; 2020; 26,
87. Ahmed, B; Dannhauser, T; Philip, N. A systematic review of reviews to identify key research opportunities within the field of ehealth implementation. J Telemed Telecare; 2019; 25,
88. Rahmani, AM; Szu-Han, W; Yu-Hsuan, K; Haghparast, M. The internet of things for applications in wearable technology. IEEE Access; 2022; 10, pp. 123579-123594. [DOI: https://dx.doi.org/10.1109/ACCESS.2022.3224487]
89. Aljedaani B, Ahmad A, Zahedi M, Babar MA. An empirical study on developing secure mobile health apps: The developers’ perspective. In: 2020 27th Asia-Pacific Software Engineering Conference (APSEC), 2020;208–217. IEEE.
© The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.