mHealth: A Strategic Field without a Solid Scientific Soul. A Systematic Review of Pain-Related Apps

Abstract

Background

Mobile health (mHealth) has undergone exponential growth in recent years. Patients and healthcare professionals are increasingly using health-related applications, at the same time as concerns about ethical issues, bias, conflicts of interest and privacy are emerging. The general aim of this paper is to provide an overview of the current state of development of mHealth.

Methods and Findings

To exemplify the issues, we made a systematic review of the pain-related apps available in scientific databases (Medline, Web of Science, Gale, Psycinfo, etc.) and the main application shops (App Store, Blackberry App World, Google Play, Nokia Store and Windows Phone Store). Only applications (designed for both patients and clinicians) focused on pain education, assessment and treatment were included. Of the 47 papers published on 34 apps in scientific databases, none were available in the app shops. A total of 283 pain-related apps were found in the five shops searched, but no articles have been published on these apps. The main limitation of this review is that we did not look at all stores in all countries.

Conclusions

There is a huge gap between the scientific and commercial faces of mHealth. Specific efforts are needed to facilitate knowledge translation and regulate commercial health-related apps.

About the Authors:

Rocío de la Vega

Affiliation: Unit for the Study and Treatment of Pain - ALGOS, Research Center for Behavior Assessment, Department of Psychology and Institut d’Investigació Sanitària Pere Virgili, Universitat Rovira i Virgili, Tarragona, Spain

Jordi Miró

* E-mail: [email protected]

Affiliation: Unit for the Study and Treatment of Pain - ALGOS, Research Center for Behavior Assessment, Department of Psychology and Institut d’Investigació Sanitària Pere Virgili, Universitat Rovira i Virgili, Tarragona, Spain

Introduction

Healthcare systems worldwide are becoming exhausted; many demands are placed on them but resources are scarce. Healthcare costs are escalating and our public health systems seem to be incapable of satisfying the needs of a fast growing population [1]. In this scenario, what is known as mobile health technology or “mHealth” – that is, healthcare supported by mobile communication technologies – has undergone exponential growth in the last few years.

Mobile health technology can make healthcare more accessible and affordable for all. It has proven to be a good way of delivering high-quality healthcare services to a variety of patient populations, particularly those with low incomes [2] and in remote places (far from reference centers) [3]. mHealth technology has also proven to be highly suitable for young people (and also very popular) [4] as they spend more time using electronic media than doing any other activity besides sleeping [5].

It has been estimated that by the end of 2016, there will be ten billion mobile devices in use around the world [3]. Patients and healthcare professionals are increasingly using health-related applications [6]. To date, more than 97,000 of these applications have been developed and in the next few years more than three million free and 300,000 paid downloads are expected to be made of mHealth applications just in the USA [7]. A recent study concluded that the Smartphone is the most popular technology among physicians since the stethoscope [1]. Furthermore, mobile phone use seems to be greater among those populations most in need of such interventions [8]. mHealth seems to be a logical, acceptable, and affordable way to extend and improve health care.

Although the progress of mHealth has many advantages, some of which have been summarized above, this extremely fast growth also has a negative side: namely, most of the procedures available have not been subject to a thorough assessment and validation [9], [10]. Explicit and sensible concerns about ethical issues, bias, conflicts of interest [11], and security and privacy problems [2] have been raised in the specialized literature.

Some action protocols and strategies are being developed to deal with these as yet unsolved issues in Europe [12], [13] and the USA [14], [15]. For example, the World Health Organization in partnership with the United Nations specialized agency for information and communication technologies has developed an initiative regarding the management of Non-Communicable Diseases using mHealth [16]. Also, some charities, and not-for-profit or private organizations have launched initiatives to boost the potentialities of mHealth. This is the case, for example, of the mHealth Alliance, hosted by the United Nations Foundation [17]. Similarly, PatientView has recently released the web page “myhealthapps.net”, recommended by the Directorate General for Communications Networks, Content and Technology of the European Commission. This web page is an evolution of the previously published “European Directory of Health Apps” [18], in which patients’ associations from all over the world used a zero-to-five Likert-type scale to rate 307 health-related apps on the extent to which they help control their condition, keep them healthy, are trustworthy, are easy to use, allow them to network with people like them/who understand them, and can be used regularly. In the context in which we find ourselves, then, commercial apps are developing exponentially, while mHealth-related scientific publications are also growing. However, it is not clear that both worlds interact and, if they do, how. That is to say, is the growth rampant, or is there fruitful interaction between the two worlds? Are research findings translated and used to improve the apps that are created or are knowledge transfer processes failing?

In this situation, it would be extremely useful if a review were to map out the terrain, identify problems and tentatively suggest avenues for improvement.

However, the field of mHealth is so wide that a complete review and analysis cannot be contemplated. Therefore, we decided to focus on pain-related apps as a way of managing an otherwise insurmountable amount of information. First, although mHealth uses various alternatives and technologies to educate patients, and to prevent and/or treat illness, apps are at the heart of the process. Two specific features of apps make it particularly important for their quality and scientific rigor to be studied: namely, (1) the app is available to consumers who do not have a professional to recommend, prescribe or even monitor how they use it, and (2) too often there is nobody “responsible” and available if the app is not working as expected or if something goes wrong. Second, we decided to concentrate on pain-related apps because pain is one of the most generalized symptoms of chronic health conditions [19]. It is a ubiquitous health problem, and well suited to be assessed and managed with these mHealth interventions [20]–[22]. So it can be readily used to explore and exemplify the issues when looking into the current state of development of mHealth.

The general aim of this paper is to provide an overview of the current state of development of mHealth. In order to do so, and to exemplify the issues, we conducted a systematic review of the pain-related apps available and reported on their characteristics; we looked both at the commercial and the scientific aspects of this development. The specific objectives of our review are to: (1) detect the number of pain-related apps reported in scientific databases, (2) find out which ones are available at the stores for general consumers, (3) identify which pain-related apps are available at the main apps shops, (4) find out which of these apps are scientifically supported, and (5) uncover any other additional support that the apps may have.

Our specific hypotheses were that (1) only a few of the apps reported in peer-reviewed publications are available to the consumer, and (2) of the apps available in the shops, very few have a scientific base.

Methods

Phase I: what can be found in scientific databases?

Search strategy and selection criteria.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [23] were followed. Data for this review were identified by searches of following scientific databases: Medline (National Library of Medicine), Science Citation Index Expanded (Web of Science), Health Reference Center Academic (Gale), Wiley Online Library, American Psychological Association (Psycinfo), SciVerse ScienceDirect (Elsevier), SpringerLink, Wolters Kluwer - Ovid - Lippincott Williams & Wilkins (CrossRef), Directory of Open Access Journals (DOAJ), Social Sciences Citation Index (Web of Science), Taylor & Francis Online - Journals, Expert Reviews (Future Science), Informa - Informa Healthcare (CrossRef), SpringerLink Open Access, Wolters Kluwer - Ovid (CrossRef), BMJ Journals, DiVA - Academic Archive Online, Informa (CrossRef), and references from relevant articles using the search terms (Pain OR *ache) AND (Smartphone OR app OR application OR electronic OR “Personal Digital Assistant” OR PDA). Only peer-reviewed articles published in English or Spanish between 1996 (the release date of the first palmtop computer [24]) and December 2013 were included.

Phase II: what scientifically assessed pain-related apps are available in the stores?

The name of each app retrieved in phase I was searched for in each of the following shops: App Store (iPhone), Blackberry App World, Google Play (Android), Nokia Store and Windows Phone Store.

Phase III: what can be found in the stores?

In December 2013, the main Smartphone application shops were reviewed: App Store (iPhone), Blackberry App World, Google Play (Android), Nokia Store and Windows Phone Store. The review was conducted in the following countries: Canada, Spain, and USA. The search terms were: “Pain”, “*ache” and “dolor”. The applications (designed for both patients and clinicians) focused on pain education, assessment and treatment were included.

Phase IV: what support do the apps available in stores have?

A step-by-step sequential strategy was followed to assess the quality of the apps found in phase III. First, the name of each app was searched for in the same databases as in Phase I. Then, the web page “myhealthapps.net” was also reviewed. All the pain-related apps were recorded. Finally, the name of each app was Google searched for such information as whether the developers had a webpage, which research centers used the app, who its creators were and/or the results it had provided, etc. This information was compared with the information obtained in phase I to see if the authors of the apps were the same as the authors of the publications.

Results

Phase I: what can be found in scientific databases?

After reviewing the databases, we found 47 papers reporting on 34 pain-related apps. Figure 1 describes our study’s selection process.

[Figure omitted. See PDF.]

Figure 1. Flow chart of our systematic review selection process.

https://doi.org/10.1371/journal.pone.0101312.g001

As can be seen in Table 1, all apps are related to assessment, and almost all are available in English (26, 76.5%) and address non-specific chronic pain problems (28, 82.4%). About two-thirds are designed for adults (22, 64.7%).

[Figure omitted. See PDF.]

Table 1. Apps reported on the scientific databases.

https://doi.org/10.1371/journal.pone.0101312.t001

Phase II: are the scientifically assessed apps available in the stores?

No pain-related app reported in any paper found during Phase I was available in any of the five main shops for the general public.

Phase III: what can be found in the stores?

A total of 283 pain-related apps were found in the five shops searched. Because of word count and space limitations, the full list is provided as an annex to the article (see Table S1).

Phase IV: what type of support do the pain-related apps available in stores have?

When we searched for these 283 apps in the scientific databases, we did not find a single article that was related to them in any way. Therefore, this search found no evidence of scientific support for the 283 pain-related apps. Nevertheless, some apps do have other support types. Figure 2 describes our app selection process.

[Figure omitted. See PDF.]

Figure 2. Flow chart of pain-related apps selection process.

https://doi.org/10.1371/journal.pone.0101312.g002

A full description of 40 apps – including name, developers, supports, pain problem it addresses, features, platform, price, language/s and user ratings – is provided as an annex to the article (see Table S2). Figure 3 summarizes the type of support that the pain-related apps have.

[Figure omitted. See PDF.]

Figure 3. Type of support that the pain-related apps have.

https://doi.org/10.1371/journal.pone.0101312.g003

Most of the apps are available in English (36, 90%), and have been developed in the USA (16, 40%), the EU (15, 37.5%), or Canada (6, 15%). The App Store and Google Play are the most important platforms, hosting 39 (97.5%) of the supported apps. The most important sources of support to these apps are: having a licensed professional as a creator (24, 62.5%) or being recommended by a patient association (12, 30%). “Pain in general” (9, 22·5%), followed by back pain (8, 20%), headache (7, 17.5%) and arthritis (6, 15%), are the types of pain that these apps are most commonly designed for. As far as the targeted consumers are concerned, most of the apps are addressed to patients (28, 70%) and only a few have been developed for healthcare professionals (5, 12.5%) or both audiences (7, 17.5%). Most patient-oriented apps provide information about the pain problem/illness and ways to check symptoms and track medication consumption. Only a few provide information about alternative ways of coping with the health problem either through videos or written instructions, for example, about exercising, massage, or even hypnosis. Professional-oriented apps provide support for diagnosis, medication dose calculation, or self-report questionnaires. All patient-oriented applications are classified as +4 years or “low maturity”, while professional-oriented are classified as +17 years.

None of the authors/developers of the apps were found to be the authors of articles about them.

Discussion

Overall, this review indicates that the commercial and scientific sides of the mHealth coin do not interact properly. We found that pain-related apps that have been reported in scientific journals have not yet made their way into the shops and are therefore unavailable to clinicians and/or patients. Conversely, 283 pain-related apps were available in the main shops, but none of them had been scientifically validated or proven to be effective. These findings are in line with our hypotheses but the situation is even more extreme than we had imagined. However, it may be just a matter of time before this state of affairs changes because some apps are currently in the last stages of the knowledge translation process. For example, Painometer V2, an app developed to help with the assessment of pain intensity is already available in Google Play and has shown some evidence of usability [72], [73] and of the psychometric properties of the scales contained [74]. Pain Squad is another app that has already reported information on usability, feasibility, and compliance [47], [48]. It is currently available in four Canadian hospitals and may be available soon at the App Store [75].

mHealth technologies have numerous important advantages over other more traditional alternatives. For example, they capture time- and date-stamped information, and provide detailed and non-biased information on such fundamental health-related variables as physical activity or physiological responses, thus reducing memory bias. They can also be extremely useful in public health actions (for example, by providing routes to help patients who have to take medications on a specific schedule) and help us reach underserved populations, those that are most in need of health care support.

In the midst of this huge, positive development there are some fundamental concerns that require appropriate responses. For example, issues of confidentiality or the protection of patients’ personal data still have to be dealt with. Furthermore, some apps occupy a “legal void”. For example, electronic diaries or cognitive-behavioral treatments for health conditions are unregulated, a situation that needs to be remedied. Overall, the results of this review indicate that consumers run some risks above and beyond paying for a potentially useless app. For example, we found some apps that claimed they could heal the body by emitting vibrations, “brain waves”, or accessing the subconscious to “tell the body to heal”. These unproven claims may lead patients to a feeling of helplessness and lack of control about their illnesses.

As mentioned above, there is a gap between the scientific and the commercial sides of the mHealth coin. Significant developments have been made in both areas but they remain essentially disconnected, advancing in parallel with no significant interaction. None of the apps in the shops have proved to have scientific support and only a fifth (57, 40+17 versions for other platforms, 20.1%) of them have some type of support. Some scientifically developed apps look promising but there is an urgent need to promote actions for knowledge translation in this field. Other researchers have found similar results when looking into other mHealth areas: apps to manage diabetes [76] and the world deadliest diseases [77]. They both found that the commercial area was significantly more developed than the research field. Referring to cardiology apps [78], they found that most of the published papers reviewed monitoring apps, but similarly to our findings, the majority was not smartphone apps themselves but computers apps that could be also used by a mobile phone or a smartphone.

In the near future, perhaps, physicians will be prescribing specific applications to specific patients for specific problems [79] (very much like today when they electronically prescribe medications, or work with the patient’s electronic clinical history system and health records). It does not make much sense for drugs to have to go through a long and complex process between the discovery of the active ingredient and being put on the market, while apps do not have to fulfill any requirements at all, not even show that they are effective and safe. There may be no need for health-related apps to go to the extremes of approved drugs, but a minimum level of quality should be compulsory. Health-related apps can also have negative effects. Therefore, we should be able to regulate what is available in stores, and prevent unregulated apps from being published in the field of health (health-related apps should inform about quality controls and prove they are efficacious before they can use the adjective health, in the same way that current laws prevent food from bearing the name “bio” if their real properties have not been subject to strict analysis). Furthermore, lists of approved health-related apps ought to be published and the general public informed, for example, through an app-related vade-mecum, so that both health experts and patients can make informed decisions about whether to use certain apps. A promising avenue that would prove fruitful in the near future is the work done by Public Agencies in the field of quality distinctions, for example, the “AppSaludable Distinctive”, reported in the last European Journal of e-practice [80] To date, and to the best of our knowledge, no pain-related app has been awarded this quality stamp and just one (Painometer v2) has applied for it [81].

Perhaps the most important limitation of this review is that we did not look at all stores in all countries. We selected three of the possibilities, not only because it was convenient, but also because it was what could be feasibly done. Our hypothesis is that if we had conducted specific reviews for the 97,000 health-related apps available worldwide, results would not have been much different, particularly considering that we explored the most important app stores and that other researchers [76]–[78] found similar results.

All the articles reviewed were related to pain assessment, with some dealing with educational issues. Future studies are needed in the area of pain management. We are aware that some research groups are working on this subject, so we can expect developments in the future. Most apps are designed for adults or adolescents, but there are very few for children. However, children are using these technologies at a very early age: 72% of children younger than eight years old use mobile devices and 50% of those use apps [82]. Therefore, additional research is greatly needed in this area if health-related apps are to be developed that are efficacious and developmentally appropriate.

Supporting Information

[Figure omitted. See PDF.]

Table S1.

Pain apps available in the main five shops.

https://doi.org/10.1371/journal.pone.0101312.s001

(DOCX)

Table S2.

Characteristics of the commercial apps that have some sort of support.

https://doi.org/10.1371/journal.pone.0101312.s002

(DOCX)

Protocol S1.

Protocol for the systematic review.

https://doi.org/10.1371/journal.pone.0101312.s003

(DOCX)

Checklist S1.

PRISMA Checklist.

https://doi.org/10.1371/journal.pone.0101312.s004

(DOC)

Acknowledgments

The authors would like to thank Carmen Muñoz and Karen Sánchez for their help in searching for the apps in the stores.

Author Contributions

Conceived and designed the experiments: RdlV JM. Analyzed the data: RdlV JM. Wrote the paper: RdlV JM.

Citation: de la Vega R, Miró J (2014) mHealth: A Strategic Field without a Solid Scientific Soul. A Systematic Review of Pain-Related Apps. PLoS ONE 9(7): e101312. https://doi.org/10.1371/journal.pone.0101312

References

1. Group TBC (2012) The Socio-Economic Impact of Mobile Health.

2. Patient Privacy in a Mobile world. A framework to address pricacy law issues in mobile health (2013)

3. West D (2012) How mobile devices are transforming healthcare. Issues Technol Innov 1–14.

4. Madden M, Lenhart A, Duggan M, Cortesi S, Gasser U (2013) Teens and technology 2013. Washington.

5. Sigman A (2012) The impact of screen media on children: a Eurovision for parliament. Improving the quality of childhood in Europe

6. Hogan NM, Kerin MJ (2012) Smart phone apps: Smart patients, steer clear. Patient Educ Couns 89: 360–361.

7. Jahns R-G (2013) The market for mHealth app services will reach $26 billion by 2017. Res Rep Available: http://www.research2guidance.com/the-market-for-mhealth-app-services-will-reach-26-billion-by-2017/. Accessed 27 November 2013.

8. Duggan M, Smith A (2013) Cell Internet Use 2013.

9. Whitehouse D, Mccormack H, Lindley J, Fernando J (2013) Editorial: mHealth Regulatory Environments. epractice.eu.

10. Sifferlin A (2013) Health Care Apps are Limited In Function. TIME.com. Available: http://healthland.time.com/2013/10/31/bad-news-about-your-favorite-health-apps-they-dont-work/. Accessed 26 November 2013.

11. Krieger WH (2013) Medical apps: public and academic perspectives. Perspect Biol Med 56: 259–273

12. Digital Agenda for Europe - European Commission (2012). Available: https://ec.europa.eu/digital-agenda/en/news/putting-patients-driving-seat-digital-future-healthcare.Accessed 26 November 2013.

13. Stylianou A, McCormack H, Kokmotou R (2013) Editorial: Applying mHealth Solutions. epractice.eu.

14. Center for Devices and Radiological Health (2013) Mobile Medical Applications. Food Drug Adm. Available: http://www.fda.gov/medicaldevices/productsandmedicalprocedures/connectedhealth/mobilemedicalapplications/default.htm. Accessed 27 November 2013.

15. Patel B (2013) Mobile Medical Applications. Guidance for Industry and Food and Drug Administration Staff.

16. International Telecommunication Union (ITU). The United Nations specialized agency for information and communication technologies. (2013) BE HE@LTHY, BE MOBILE. Available: http://www.itu.int/en/ITU-D/ICT-Applications/eHEALTH/Pages/Be_Healthy.aspx.Accessed 27 November 2013.

17. mHealth Alliance webpage (n.d.). Available: http://mhealthalliance.org/. Accessed 26 November 2013.

18. Madelin R (2013) European Directory of Health Apps 2012–2013.

19. Goldberg DS, McGee SJ (2011) Pain as a global public health priority. BMC Public Health 11: 770

20. Bender JL, Radhakrishnan A, Diorio C, Englesakis M, Jadad AR (2011) Can pain be managed through the Internet? A systematic review of randomized controlled trials. Pain 152: 1740–1750

21. Williams D a (2011) Web-based behavioral interventions for the management of chronic pain. Curr Rheumatol Rep 13: 543–549

22. Rosser B a, Eccleston C (2011) Smartphone applications for pain management. J Telemed Telecare 17: 308–312

23. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097

24. Palmtop computer release date (n.d.). Available: http://en.wikipedia.org/wiki/Palm_(PDA). Accessed 24 November 2013.

25. Aaron La, Turner Ja, Mancl La, Sawchuk CN, Huggins KH, et al. (2006) Daily pain coping among patients with chronic temporomandibular disorder pain: an electronic diary study. J Orofac Pain 20: 125–137.

26. Aaron LA, Turner JA, Mancl L, Brister H, Sawchuk CN (2005) Electronic diary assessment of pain-related variables: Is reactivity a problem? J Pain 6: 107–115.

27. Turner JA, Mancl L, Aaron LA (2005) Brief cognitive-behavioral therapy for temporomandibular disorder pain: Effects on daily electronic outcome and process measures. Pain 117: 377–387.

28. Affleck G, Urrows S, Tennen H, Higgins P, Abeles M (1996) Sequential daily relations of sleep, pain intensity, and attention to pain among women with fibromyalgia. Pain 68: 363–368.

29. Allena M, Cuzzoni MG, Tassorelli C, Nappi G, Antonaci F (2012) An electronic diary on a palm device for headache monitoring: a preliminary experience. J Headache Pain 13: 537–541

30. Alfvén G (2010) SMS pain diary: a method for real-time data capture of recurrent pain in childhood. Acta Paediatr 99: 1047–1053

31. Chan SS, Chu CP, Cheng BC, Chen PP (2004) Data management using the personal digital assistant in an acute pain service. Anaesth Intensive Care 32: 81–86.

32. Connelly M, Miller T, Gerry G, Bickel J (2010) Electronic momentary assessment of weather changes as a trigger of headaches in children. Headache 50: 779–789

33. Connelly M, Anthony KK, Sarniak R, Bromberg MH, Gil KM, et al. (2010) Parent pain responses as predictors of daily activities and mood in children with juvenile idiopathic arthritis: the utility of electronic diaries. J Pain Symptom Manage 39: 579–590

34. Connelly M, Bromberg MH, Anthony KK, Gil KM, Franks L, et al. (2011) Emotion Regulation Predicts Pain and Functioning in Children With Juvenile Idiopathic Arthritis: An Electronic Diary Study. J Pediatr Psychol 37: 43–52

35. Evans SR, Simpson DM, Kitch DW, King A, Clifford DB, et al. (2007) A Randomized Trial Evaluating Prosaptide(TM) for HIV-Associated Sensory Neuropathies: Use of an Electronic Diary to Record Neuropathic Pain: e551 - ProQuest. PLoS One 2

36. Gaertner J, Elsner F, Pollmann-Dahmen K, Radbruch L, Sabatowski R (2004) Electronic pain diary: a randomized crossover study. J Pain Symptom Manage 28: 259–267.

37. Ghinea G, Spyridonis F, Serif T, Frank aO (2008) 3-D pain drawings-mobile data collection using a PDA. IEEE Trans Inf Technol Biomed a Publ IEEE Eng Med Biol Soc 12: 27–33

38. Goldberg J, Wolf A, Silberstein S, Gebeline-Myers C, Hopkins M, et al. (2007) Evaluation of an electronic diary as a diagnostic tool to study headache and premenstrual symptoms in migraineurs. Headache 47: 384–396

39. Goldstein HS, Rabaza JR, Gonzalez M, Verdeja JC (2003) Evaluation of pain and disability in plug repair with the aid of a personal digital assistant. Hernia J hernias Abdom wall Surg 7: 25–28

40. Gulur P, Rodi SW, Washington Ta, Cravero JP, Fanciullo GJ, et al. (2009) Computer Face Scale for measuring pediatric pain and mood. J pain 10: 173–179

41. Heiberg T, Kvien TK, Dale Ø, Mowinckel P, Aanerud GJ, et al. (2007) Daily health status registration (patient diary) in patients with rheumatoid arthritis: a comparison between personal digital assistant and paper-pencil format. Arthritis Rheum 57: 454–460

42. Jacob E, Stinson J, Duran J, Gupta A, Gerla M, et al. (2012) Usability testing of a Smartphone for accessing a web-based e-diary for self-monitoring of pain and symptoms in sickle cell disease. J Pediatr Hematol Oncol 34: 326–335

43. Jacob E, Duran J, Stinson J, Lewis MA, Zeltzer L (2013) Remote monitoring of pain and symptoms using wireless technology in children and adolescents with sickle cell disease. J Am Assoc Nurse Pract 25: 42–54

44. Jamison RN, Gracely RH, Raymond SA, Levine JG, Marino B, et al. (2002) Comparative study of electronic vs. paper VAS ratings: a randomized, crossover trial using healthy volunteers. Pain 99: 341–347.

45. Jamison RN, Raymond SA, Levine JG, Slawsby EA, Nedeljkovic SS, et al. (2001) Electronic diaries for monitoring chronic pain: 1-year validation study. Pain 91: 277–285.

46. Jamison RN, Raymond SA, Slawsby EA, McHugo GJ, Baird JC (2006) Pain Assessment in Patients With Low Back Pain: Comparison of Weekly Recall and Momentary Electronic Data. J Pain 7: 192–199.

47. Jibb L, Stinson J, Nathan P, Maloney A, Dupuis L, et al. (2012) Pain Squad: usability testing of a multidimensional electronic pain diary for adolescents with cancer. J Pain 13.

48. Stinson J, Jibb LA, Nguyen C, Nathan PC, Maloney AM, et al. (2013) Development and Testing of a Multidimensional iPhone Pain Assessment Application for Adolescents with Cancer. J Med Internet Res 15: e51

49. Johnson KB, Luckmann R, Vidal A (2010) Design of a handheld electronic pain, treatment and activity diary. J Biomed Inform 43: S32–S36.

50. Junker U, Freynhagen R, Längler K, Gockel U, Schmidt U, et al. (2008) Paper versus electronic rating scales for pain assessment: a prospective, randomised, cross-over validation study with 200 chronic pain patients.

51. Kristjánsdóttir Ó, Fors Ea, Eide E, Finset A, van Dulmen S, et al. (2011) Written online situational feedback via mobile phone to support self-management of chronic widespread pain: a usability study of a Web-based intervention. BMC Musculoskelet Disord 12: 51

52. Kristjánsdóttir Ó, Fors EA, Eide E, Finset A, Stensrud TL, et al. (2013) A smartphone-based intervention with diaries and therapist feedback to reduce catastrophizing and increase functioning in women with chronic widespread pain. part 2: 11-month follow-up results of a randomized trial. J Med Internet Res 15: e72

53. Kristjánsdóttir Ó, Fors E, Eide E, Finset A, Stensrud T, et al. (2013) A Smartphone-Based Intervention With Diaries and Therapist-Feedback to Reduce Catastrophizing and Increase Functioning in Women With Chronic Widespread Pain: Randomized Controlled Trial. J Med Internet Res 15: e5

54. Lewandowski AS, Palermo TM, Kirchner HL, Drotar D (2009) Comparing Diary and Retrospective Reports of Pain and Activity Restriction in Children and Adolescents with Chronic Pain Conditions. Clin J Pain 25: 299–306

55. Marceau LD, Link CL, Smith LD, Carolan SJ, Jamison RN (2010) In-Clinic Use of Electronic Pain Diaries: Barriers of Implementation Among Pain Physicians. J Pain Symptom Manage 40: 391–404.

56. McClellan CB, Schatz JC, Puffer E, Sanchez CE, Stancil MT, et al. (2009) Use of handheld wireless technology for a home-based sickle cell pain management protocol. J Pediatr Psychol 34: 564–573

57. Palermo TM, Valenzuela D, Stork PP (2004) A randomized trial of electronic versus paper pain diaries in children: impact on compliance, accuracy, and acceptability. Pain 107: 213–219

58. Peters ML, Sorbi MJ, Kruise DA, Kerssens JJ, Verhaak PF, et al. (2000) Electronic diary assessment of pain, disability and psychological adaptation in patients differing in duration of pain. Pain 84: 181–192.

59. Roelofs J, Peters ML, Patijn J, Schouten EGW, Vlaeyen JWS (2004) Electronic diary assessment of pain-related fear, attention to pain, and pain intensity in chronic low back pain patients. Pain 112: 335–342.

60. Roelofs J, Peters ML, Patijn J, Schouten EGW, Vlaeyen JWS (2006) An electronic diary assessment of the effects of distraction and attentional focusing on pain intensity in chronic low back pain patients. Br J Health Psychol 11: 595–606

61. Sorbi MJ, Peters ML, Kruise DA, Maas CJM, Kerssens JJ, et al. (2006) Electronic momentary assessment in chronic pain I: psychological pain responses as predictors of pain intensity. Clin J Pain 22: 55–66.

62. Sorbi MJ, Peters ML, Kruise DA, Maas CJM, Kerssens JJ, et al. (2006) Electronic momentary assessment in chronic pain II: pain and psychological pain responses as predictors of pain disability. Clin J Pain 22: 67–81.

63. Sorbi MJ, Mak SB, Houtveen JH, Kleiboer AM, van Doornen LJP (2007) Mobile Web-based monitoring and coaching: feasibility in chronic migraine. J Med Internet Res 9: e38

64. Kleiboer A, Sorbi M, Mérelle S, Passchier J, van Doornen L (2009) Utility and preliminary effects of online digital assistance (ODA) for behavioral attack prevention in migraine. Telemed J e-health Off J Am Telemed Assoc 15: 682–690

65. Stinson J, Petroz GC, Tait G, Feldman BM, Streiner D, et al. (2006) e-Ouch: usability testing of an electronic chronic pain diary for adolescents with arthritis. Clin J Pain 22: 295–305

66. Stinson JN, Petroz GC, Stevens BJ, Feldman BM, Streiner D, et al. (2008) Working out the kinks: testing the feasibility of an electronic pain diary for adolescents with arthritis. Pain Res Manag 13: 375–382.

67. Stinson J, Stevens BJ, Feldman BM, Streiner D, McGrath PJ, et al. (2008) Construct validity of a multidimensional electronic pain diary for adolescents with arthritis. Pain 136: 281–292

68. Stone AA, Broderick JE, Schwartz JE, Shiffman S, Litcher-Kelly L, et al. (2003) Intensive momentary reporting of pain with an electronic diary: reactivity, compliance, and patient satisfaction. Pain 104: 343–351.

69. VanDenKerkhof EG, Goldstein DH, Lane J, Rimmer MJ, Van Dijk JP (2003) Using a personal digital assistant enhances gathering of patient data on an acute pain management service: a pilot study. Can J Anaesth 50: 368–375

70. Walker L, Sorrells S (2002) Brief report: Assessment of children’s gastrointestinal symptoms for clinical trials. J Pediatr Psychol 27: 303–307.

71. Wood C, von Baeyer CL, Falinower S, Moyse D, Annequin D, et al. (2011) Electronic and paper versions of a faces pain intensity scale: concordance and preference in hospitalized children. BMC Pediatr 11: 87

72. De la Vega R, Castarlenas E, Roset R, Sánchez-Rodríguez E, Solé E, et al.. (2013) Testing Painometer: an App to Assess Pain Intensity. International Forum on Pediatric Pain. Halifax, Canada.

73. De La Vega R, Sánchez-Rodríguez E, Castarlenas E, Roset R, Tomé-Pires C, et al.. (2013) Painometer: an app to assess pain intensity. International Symposium on Pediatric Pain. Stockholm.

74. Castarlenas E, Sánchez-Rodríguez E, de la Vega R, Roset R, Miró J (in press) Agreement between verbal and electronic versions of the Numerical Rating Scale (NRS-11) when used to assess pain intensity in adolescents. Clin J Pain

75. Wong C (2013) Pain Squad App Gamifies Health Care In Canada And Beyond. CommerceLab. Available: http://www.commercelab.ca/a-checkup-on-pain-squad-the-canadian-app-that-gamified-health-care/. Accessed 24 November 2013.

76. Goyal S, Cafazzo Ja (2013) Mobile phone health apps for diabetes management: current evidence and future developments. QJM 106: 1067–1069

77. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M, Sainz-De-Abajo B (2014) Comparison of Mobile Apps for the Leading Causes of Death Among Different Income Zones: A Review of the Literature and App Stores. JMIR mhealth uhealth 2: e1

78. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M, Herreros-González J (2013) Mobile Apps in Cardiology: Review. JMIR mhealth uhealth 1: e15

79. Wiltfong J (2013) One Quarter (26%) Globally, Who Use Medical Apps, Say Recommendation Came From Medical Professional. Ipsos. Available: http://www.ipsos-na.com/news-polls/pressrelease.aspx?id=6318. Accessed 24 November 2013.

80. Ferrero Álvarez Rementería J, López Santana V, Escobar Ubreva A, Vázquez-Vázquez M, Rodríguez Contreras H, et al. (2013) Quality and Safety Strategy for Mobile Health Applications: A Certification Programme. Eur J ePractice 20.

81. Andalusian Quality Agency (2013) Distintivo AppSaludable - Catálogo de aplicaciones. Available: http://www.calidadappsalud.com/distintivo/catalogo. Accessed 24 November 2013.

82. Rideout V, Saphir M (2013) Zero to Eight: Children’s Media Use In America 2013.