Introduction
Background
Multiple sclerosis (MS) is one of the world’s most common neurologic disorders leading to severe disability in young adults. More than 2.3 million people live with MS in the world, with higher incidences in Northern Europe and in temperate climates [1]. MS is a chronic condition with high self-management needs [2] that require significant support [3-5] since the majority of self-management occurs at home [6]. MS symptoms range from simple visual disturbances and altered sensation to severe fatigue and cognitive problems with mobility issues [2]. Persons with MS can also go through stretches of periods in which symptoms worsen, called “attacks” or “relapses” [1,2]. Fatigue is the most common and disabling symptom of MS, affecting up to 80% of patients [7-12]. There seems to be some association between MS fatigue and environmental conditions such as the time of the day and weather [13] but it is still unknown how much of an influence these factors have. MS fatigue has a huge impact on quality of life [8,9] and socioeconomic status for persons with MS and is the major reason for early retirement [14]. As a result of MS, persons with MS are typically less active [15] and have reduced levels of physical activity [16-18].
Mobile health (mHealth) is the delivery of health care or health care–related services through the use of portable devices [19]. The use of mHealth software apps has grown in recent years, to the point where commercial app stores hold thousands of health care–related apps [20]. Several digital and remote communication technology apps have been developed for MS clinical monitoring and management, to complement traditional clinical approaches [21]. Wearable devices, including actigraphy, gyroscopes, and body temperature or heart rate monitors, have been used to obtain a more comprehensive assessment of different body functions and to monitor disability in persons with MS [22]. A recent review [21] showed that the use of apps for MS as complements to traditional in-clinic care can improve outcomes and increase access to care, disease information, and support. The use of mHealth could help persons with MS to be more active in their self-management, for example, by tracking adherence to treatment, changes in bladder and bowel habits, and activity and mood.
However, digital health technologies have low adoption rates by patients with chronic diseases, in part explained by an inappropriate fit of these technologies with patients’ daily lives and the high patient burden associated with digital self-monitoring [23]. Studies regarding the attitudes of persons with MS toward using smartphone apps highlight the benefits of tailoring apps to specific patient needs [24,25]. Further, involving patients and health care professionals in the design of technology is a need that has been often raised [26-32] and could be helpful in increasing adoption rates [33-35].
In previous studies, we explored available mHealth apps for persons with MS and found that only a handful exist [31,36]. This prompted the user-centered design (UCD) process of an mHealth solution specifically tailored for this target population called More Stamina.
User-Centered Design
UCD is a design philosophy that places the needs and characteristics of end users in the center of software design and development [37-39]. Through the use of UCD, solutions that are specific to the characteristics of the intended users are designed with higher acceptance and fewer user errors [38-40]. The overall process of UCD comprises the specification of the context of use (understand users and their characteristics and environment), specification of the requirements (identify the granular requirements and needs), production of solutions (start an iterative process of design and development), and evaluation (testing to find critical feedback on the product) [37,41].
The evaluation of usability entails a wide array of methodologies that vary in terms of research design, complexity, cost, and duration [42]. Different methods can be used to evaluate a system design on its usability, such as expert-based inspections and user-based testing [43]. Following UCD ensures that mHealth apps are more likely to meet end user needs and expectations [44,45].
More Stamina
More Stamina is a gamified task organization tool designed to help persons with MS manage their energy, to minimize the impact of fatigue on their daily life. The tool acts as a to-do list where users can input the tasks they want to accomplish that day in a simple manner (Figure 1). More Stamina is currently available in English, Spanish, Finnish, and German.
The overall concept is that a person’s energy is represented through a visual metaphor (progress bar) and a symbolic unit that quantifies the amount of estimated effort a given activity might take (Stamina Credits). Users start their day with 100 Stamina Credits and assign them to new activities for that day. As users go through their day and complete the activities, they mark them as done, and the system prompts them to assess whether their effort was underestimated, overestimated, or properly estimated. More Stamina keeps track of these answers as data points and starts analyzing and creating a trend for each activity (eg, “shopping”). Repeated use of More Stamina allows it to learn about the user’s habits, and once sufficient information is gathered on “shopping,” a recommendation feature starts reminding users of their own tendencies.
The app logs user activity through the in-built sensors of the device and is able to monitor the number of steps, walking pace, distances, and GPS positioning. Patient-reported outcomes are available and optional in More Stamina. Standardized tools such as the Fatigue Severity Scale [46] and Chalder Fatigue Scale [47] are loaded into the system. Usage statistics are gathered locally for each added activity to keep track and collect assessments; the user can choose to share these statistics to a secure server for analysis. Users have control as to which information to disclose and with whom, whether it’s personal, clinical, or treatment-related. Additionally, they can opt-in to send de-identified information for research purposes.
Current Status
The More Stamina app was created following the UCD approach through iterative development, allowing continuous improvement of the app. The progress of the solution through the different phases of the UCD process is reported in our previous studies. The state of the practice of health apps for MS was studied through a systematic app review [31,36]. The needs, barriers, and facilitators of mHealth apps for persons with MS were explored using focus groups and interviews [25]. User profiles, or “personas,” were created to aid the design process [25,48], and the design process, prototyping, and initial usability testing have been described [49].
The work intended in this phase will assess the feasibility, acceptability, and usability of the mHealth solution with persons with MS (see Figure 2).
Enlarge this image.Figure 2. Phases of the user-centered design for the More Stamina app. The green boxes indicate the areas of focus for the current study. mHealth: mobile health; MS: multiple sclerosis.
Objectives
The overall study aims are to explore the feasibility, acceptability, and usability of More Stamina, a mobile app for fatigue self-management for persons with MS.
The specific objectives are to estimate adherence to the use of More Stamina; estimate the effect of More Stamina adherence on behavior change, measured through changes in the amount of activities and amount of estimated energy per activity; estimate the effect of adherence to the use of More Stamina on the perception of fatigue management; validate the value proposition of More Stamina; identify More Stamina user activity patterns of persons with MS; and identify factors associated with the use of More Stamina.
Methods
Study Design
A mixed-methods, multicenter study will be used to assess the feasibility, acceptability, and usability of More Stamina. The study will take place during the third and fourth quarters of 2020 (Q3-Q4 2020) in 3 locations: Argentina, Spain, and Switzerland.
A longitudinal cohort pilot study will be conducted using a series of well-established standardized tools for user-based evaluations and surveys (see Standardized Tools). Participants’ background information regarding health information, quality of life, and familiarity with mobile technologies will be collected. Potential cultural differences will also be explored through qualitative approaches. System usage logs will quantify patient engagement with the system [50].
Settings
This study is part of a collaborative project between researchers and different institutions across the globe. The work will take place in 3 locations with local teams. Sessions will be conducted by a native language facilitator. Overall study coordination will be carried out by Dr. Guido Giunti from the University of Oulu as part of the More Stamina research project.
Argentina
Hospital Italiano de Buenos Aires (HIBA) is a university general hospital in the Autonomous City of Buenos Aires that includes 2 hospitals (Central Hospital and Hospital Italiano de San Justo Agustin Rocca) and 22 primary care centers. This organization has 750 beds, conducts approximately 2.5 million outpatient visits per year, and includes a health maintenance organization that delivers prepaid health care to approximately 150,000 members per year.
The Argentinian local team will be composed of researchers from HIBA.
Spain
The Vithas Nisa Sevilla Hospital (VNH) ranks as the number one private center in Andalusia and fourth in all of Spain with a specialized unit aimed at researching and treating MS. This unit is composed of a multidisciplinary team, including neurologists, neuropsychologists, physical therapists, clinical researchers, nurses, and administrative staff.
The Universidad de Sevilla is the main house of learning in the Andalusian province of Spain and provides superior education by means of studies, teaching, and research, as well as the generation, development, and diffusion of knowledge to serve citizens and society.
The Spanish local team will be composed of researchers from the Universidad de Sevilla and VNH.
Switzerland
Kliniken Valens is a center specializing in neurological rehabilitation services located in Valens, Switzerland. Kliniken Valens employs a multidisciplinary staff, including neurologists, physiotherapists, occupational therapists, speech therapists, and sports therapists. In 2019, a total of 550 persons with MS were admitted for neurological rehabilitation, of which 70% suffered from fatigue and this was the primary focus of their stay.
The Swiss local team will be composed of researchers from Kliniken Valens and the University of Oulu.
Recruitment and Sample Size
Persons with MS will be recruited from patient databases at HIBA, VNH, and Kliniken Valens and invited to participate in the study. Inclusion criteria will require each participant to be > 18 years old, have received a confirmed MS diagnosis according to McDonalds criteria [51] at least 1 year prior to the study, have none to moderate physical disability (Expanded Disability Status Scale [EDSS] < 6.5) at the time of recruitment, have no major cognitive or haptile impairment influencing the ability to use the app, be fluent in one of the languages in which More Stamina is available, and be the owner or user of a compatible smartphone device with internet access.
Exclusion criteria include refusal to participate, cognitive or physical impairment that prevents the use of mobile phones, and inability to attend the follow-up encounters.
This study seeks to enroll at least 20 patients that meet the criteria from each site for the longitudinal cohort study (total n=60). The sample size is an adequate number because of the methods used that provide extensive, detailed data [52].
Standardized Tools
The following is a description of the standardized tools that are used in this study. Language-appropriate and culturally validated versions will be used accordingly.
Expanded Disability Status Scale
The EDSS is a method of quantifying disability in MS and monitoring changes in the level of disability over time. It is widely used in clinical trials and the assessment of persons with MS [53]. The EDSS scale ranges from 0 to 10 in 0.5-unit increments that represent higher levels of disability. Scoring is based on an examination by a neurologist.
Quality of Life in Neurological Disorders
Neuro-QoL (Quality of Life in Neurological Disorders) is a measurement system that evaluates and monitors the physical, mental, and social effects experienced by adults and children living with neurological conditions [54].
Chalder Fatigue Scale
The Chalder Fatigue Scale is a self-administered questionnaire for measuring the extent and severity of fatigue within both clinical and nonclinical epidemiological populations. Although originally developed to measure the extent of chronic fatigue symptoms within clinical populations, the scale was revised and is now more widely used to measure the severity of “tiredness” rather than just chronic fatigue syndrome [47].
eHealth Literacy Scale
The eHealth Literacy Scale (eHeals) is an 8-item scale that tends to measure perceived skills at finding, evaluating, and applying electronic health information to health problems [55]. The instrument has been proven to be a reliable and easy-to-use self-report tool and has been used in some studies. The scale is based on a model that distinguishes between 6 types of literacy skills: traditional literacy, health literacy, information literacy, scientific literacy, computer literacy, and media literacy. Accordingly, the eHeals aims to measure a broad overview of literacy skills, which might make it a potential instrument to assess the effects of electronic health literacy–tailored strategies to deliver online information and apps.
System Usability Scale
The System Usability Scale provides a “quick and dirty,” reliable tool for measuring a product’s usability [56]. It consists of a 10-item short questionnaire with 5 response options for respondents, ranging from “Strongly agree” to “Strongly disagree.”
Think-Aloud Protocol
Think-aloud is a well-established technique for usability assessment, commonly used to determine users’ thoughts and opinions while they perform a list of specified tasks with a system [57]. A think-aloud protocol asks users to express their immediate thoughts and reactions during their interactions with a system. The sessions are normally recorded, or notes are taken [58]. Minimal intervention from the usability tester assures users’ thought processes are not interrupted except to remind them to keep talking. The focus is on understanding users’ decision-making processes and on how users experience the system in their own words. Only a small sample of users is needed due to the extensive, detailed data it provides [59,60].
To guide the usability evaluation, case scenarios were created for what would be considered as normal everyday user interactions with the More Stamina solution. A panel of neurologists, physiotherapists, and designers validated the different scenarios. The specific scenarios patients will have to perform will consist of (1) completing their user profile, (2) creating a new activity for themselves, (3) managing previously recorded activities, (4) planning for a future activity, and (5) responding to one of the integrated surveys (Multimedia Appendix 1).
Participants will be instructed to assess the More Stamina app using the think-aloud method, stating out loud what they are doing, what thoughts come to their minds, and how are they interacting with the app. The facilitator will record comments and actions of the participants with screen-capturing software and an audio recorder.
Open-Ended Interviews
Qualitative inquiries are useful to provide insight into complex and multifaceted experiences of individuals when a rich description is the main goal of the study [61]. Participants of our study will be interviewed with open-ended questions to explore their experiences with the mHealth solution. They will be asked to comment on sections of the system they thought were well designed, to comment on sections that were inadequately designed, and to provide any further comments they might have about system usability.
User Behavior
Within More Stamina, Stamina Credits are a numeric continuous variable that ranges from 0 to 100 to represent the estimated effort of an activity. This variable is used to gauge how effective the user is in predicting the effort of each individual activity.
User activity patterns over time will be aggregated per week and described. Adherence will be estimated in each follow-up meeting. Adherence with app use will be estimated using the frequency of weekly use throughout the tracking period.
Study Flow
Persons with MS from each location will be invited to be part of the study and go through the process of informed consent where they will be briefed about the overall study and their rights as participants (see Ethical Considerations). After providing informed consent, participants will be part of a series of 4 workshops. Prospective monitoring will be carried out for 2 months for all participants at all sites, and daily measurements of More Stamina during that period will be considered.
Encounter 1 will be used to set up user accounts for participants, introduce the More Stamina solution, and install it on their smartphones. Monitoring will begin after the installation of More Stamina. All participants will complete the following standardized tools: EDSS, NeuroQoL, Chalder Fatigue Scale, eHeals, and System Usability Scale.
A subgroup of up to 5 participants per site (total n=15) will be selected for convenience to represent the main themes of this study: time since diagnosis (prolonged or recent), digital health literacy (expert user or beginner user), and disease burden (high or low). This subgroup will go through the think-aloud protocol by performing the specific scenarios and be interviewed about ease, utility, and perceived benefit. The sessions will be digitally audio recorded and video recorded, capturing on-screen navigation on screens.
There will be 3 face-to-face follow up meetings: day 15 (Encounter 2), day 30 (Encounter 3), and at the end of the follow-up at day 60 (Encounter 4). In these meetings, feedback will be obtained on the use of the More Stamina app regarding its functionality, usability problems, daily use, and information concerning disability, quality of life, and adherence. The overall study flow that will be replicated in each site is shown in Figure 3.
Data Analysis
Quantitative Analysis
Descriptive statistics will be used to summarize the participants’ background and characteristics. Categorical variables will be presented as absolute and relative frequencies. Continuous variables will be presented as mean and standard deviation or median with interquartile range depending on distribution. A P value < 5% will be considered statistically significant. Statistical analysis will be performed using STATA v15.
Quantitative analysis will be performed on the data collected through the mHealth solution. A linear regression model will be used to assess the association between the degree of adherence to the app, changes in the amount of activity, and changes in the amount of estimated stamina per activity. The outcome variables will be changes between baseline and final activities, changes in estimated stamina per activity for specific activities selected by the researcher team based on frequency and relevance, and changes in fatigue self-perception. Changes will be calculated considering the baseline measurement and final measurement after the 3-month tracking period. A multivariate linear regression model adjusted for age, sex, and eHEALS will be used. The need for inclusion of other potential confounders will be evaluated.
Participants who are above the median usage of the app will be defined as heavy users. The median is less affected by outliers and skewed data and is usually the preferred measure of central tendency when the distribution is not symmetrical. Potential factors associated with the use of the app will be explored with a univariate logistic regression model.
Qualitative analysis will be performed on the data collected through the think-aloud protocol, individual interviews, and focus groups by using thematic analysis to identify emerging themes [62]. Each transcript will be reviewed and coded independently by 2 researchers. The themes will be combined by agreement of the 2 researchers, involving a third researcher in the event of a disagreement. Major themes and subthemes will be developed via an iterative review process. In order to help ensure the integrity of the content analyses, the guidelines recommended by Shenton [63] will be followed, which include collecting and analyzing data in an iterative process to identify themes and generating an audit trail, among others.
Crosscultural Aspects
Cultural contexts influence the way users choose, utilize, and conceptualize products and technology [64]. According to Evers and Day [65], culture is a discernible variable in interface acceptance, and it seems to play some role in perceiving subjective preferences in design [66]. Crosscultural partnerships can positively influence product development, but the use of traditional usability testing techniques in crosscultural settings can be problematic and may produce unexpected or spurious results [67].
Qualitative and quantitative data will be explored, paying attention to potential differences that pertain to cultural and geographical contexts such as traditions, habits, and weather conditions. In order to mitigate the risk of spurious results in usability testing due to crosscultural settings, we will use native speakers who are in a trusted position related to the participants.
Data Handling
Access to personal information will be restricted to the investigators of the study, health authorities, the Research Ethics Committee, and the monitors and auditors of the study. They will be subject to the duty of secrecy inherent to their profession, when necessary, to verify the data and procedures of the study, but always maintaining the confidentiality of the same according to the current legislation. Participants may exercise their rights of access, rectification, cancellation, and opposition of data according to the European Union General Data Protection Regulations [68].
The information and personal data of the participants will be kept in a completely confidential form with all the rigor of the law. Participants will be asked to not use any names during group discussions. Reports of study results will not include any identifying information. The paper questionnaires used will be digitized. The audio recordings of the group discussions will be typed and kept in secure servers at the University of Oulu. After the transcriptions are finished, the audio recordings will be destroyed. The typed transcripts will remain on password-protected computers, and any hard copy will be kept in a closed filing cabinet. Only members of the research team will be able to listen to the recordings or read the typed versions.
Ethical Considerations
The ethical research guidelines of the University of Oulu [69] and University of Seville [70] will be followed. Ethical approval will be obtained to ensure that the research is done in accordance with the Declaration of Helsinki and in line with the current local legislations from the respective authorities: HIBA’s Institutional Review Board (Argentina), the Ethics Committee of the Regional Ministry of Health of the Government of Andalusia (Spain), and the Swiss Ethics Committee on Research Involving Humans (Switzerland). The study has been registered in ClinicalTrials.gov with the identifier NCT04244214.
The participants will be informed about the nature of the research project; the reasons for their participation; risks, benefits, and alternatives associated with the research; and their rights as research subjects before agreeing to participate. Steps will be taken to ensure that data gathered from participants will be kept under strict security, anonymity, and privacy.
Results
The study will take place during the third and last quarters of 2020 (Q3-Q4 2020). Outcomes will be published in peer-reviewed medical journals and presented at international conferences.
Discussion
Overview
This protocol presents the work intended to take place to assess the feasibility, acceptability, and usability of a fatigue management mHealth solution for persons with MS.
Persons with MS have a different attitude than other people towards physical activity [18] and are typically less active than healthy persons [15]. The proposed approach of More Stamina for fatigue management is in line with common approaches in energy conservation education programs and fatigue management for MS [71,72]. The goal is to help the patient save energy through the implementation of different strategies such as work simplification or the use of task prioritization.
By tracking and collecting use and contextual information, the present study will also help to understand underlying factors and causes for MS fatigue.
The goal of this pilot study is to further our understanding of the potential issues and challenges that will be used as the foundations for a larger randomized control study.
Limitations
It is possible that the differences between the patient group who participated in the UCD and the target groups of this study are significant enough that there is a preference mismatch. For example, the importance of fatigue management could vary depending on cultural factors or adherence, or interest could even be radically different depending on economic factors. This is expected, and understanding potential differences is part of the aims of the study. Finally, the nature of the study does not allow the assessment of health outcomes due to the lack of randomized exposure and monitoring period. Such explorations will be the subject of future studies.
Acknowledgments
The work on this project is funded by the University of Oulu’s Innovation Center grant for the More Stamina project. The present study is also conducted in collaboration with the “Understanding Daily Multiple Sclerosis Related Fatigue: A Participatory Health Informatics Approach” (MSF-PHIA) project, funded by the Fondo Europeo de Desarrollo Regional (FEDER) and the Andalusian Government from Spain (US-1263715).
Additionally, the European Union Horizon 2020 Marie Skłodowska-Curie Action CHESS ITN (grant agreement No. 676201), European Network for the Joint Evaluation of Connected Health Technology (ENJECT) (COST Action TD1405), and FinCEAL Plus BRIDGES program from the Finnish Ministry of Education and Culture funded previous activities for this project.
We would like to thank Prof. Minna Isomursu, Peter Oesch, PhD, Analia Baum, MD, and Fernando Testa for their cooperation and support.
Conflicts of Interest
None declared.
Multimedia Appendix 1
Case scenarios.
Browne P, Chandraratna D, Angood C, Tremlett H, Baker C, Taylor BV, et al. Atlas of Multiple Sclerosis 2013: A growing global problem with widespread inequity. Neurology 2014 Sep 09;83(11):1022-1024
Goldenberg M. Multiple sclerosis review. P T 2012 Mar;37(3):175-184
Lorig K, Holman H. Self-management education: history, definition, outcomes, and mechanisms. Ann Behav Med 2003 Aug;26(1):1-7.
Schulman-Green D, Jaser S, Martin F, Alonzo A, Grey M, McCorkle R, et al. Processes of self-management in chronic illness. J Nurs Scholarsh 2012 Jun;44(2):136-144
Mirkovic J, Kristjansdottir OB, Stenberg U, Krogseth T, Stange KC, Ruland CM. Patient Insights Into the Design of Technology to Support a Strengths-Based Approach to Health Care. JMIR Res Protoc 2016 Aug 24;5(3):e175
Anderson RM, Funnell MM. Patient empowerment: myths and misconceptions. Patient Educ Couns 2010 Jun;79(3):277-282
Krupp L. Fatigue is intrinsic to multiple sclerosis (MS) and is the most commonly reported symptom of the disease. Mult Scler 2006 Aug 02;12(4):367-368.
Pittion-Vouyovitch S, Debouverie M, Guillemin F, Vandenberghe N, Anxionnat R, Vespignani H. Fatigue in multiple sclerosis is related to disability, depression and quality of life. J Neurol Sci 2006 Apr 15;243(1-2):39-45.
Janardhan V, Bakshi R. Quality of life in patients with multiple sclerosis. Journal of the Neurological Sciences 2002 Dec;205(1):51-58.
Fisk JD, Pontefract A, Ritvo PG, Archibald CJ, Murray T. The Impact of Fatigue on Patients with Multiple Sclerosis. Can. j. neurol. sci 2015 Sep 18;21(1):9-14.
Zwibel HL. Contribution of impaired mobility and general symptoms to the burden of multiple sclerosis. Adv Ther 2009 Dec 16;26(12):1043-1057.
Hadjimichael O, Vollmer T, Oleen-Burkey M, North American Research Committee on Multiple Sclerosis. Fatigue characteristics in multiple sclerosis: the North American Research Committee on Multiple Sclerosis (NARCOMS) survey. Health Qual Life Outcomes 2008 Nov 14;6(1):100
Mollaoğlu M, Ustün E. Fatigue in multiple sclerosis patients. J Clin Nurs 2009 May;18(9):1231-1238.
Simmons RD, Tribe KL, McDonald EA. Living with multiple sclerosis: longitudinal changes in employment and the importance of symptom management. J Neurol 2010 Jun;257(6):926-936.
Motl RW, McAuley E, Snook EM. Physical activity and multiple sclerosis: a meta-analysis. Mult Scler 2005 Aug 02;11(4):459-463.
Freal JE, Kraft GH, Coryell JK. Symptomatic fatigue in multiple sclerosis. Arch Phys Med Rehabil 1984 Mar;65(3):135-138.
Ponichtera-Mulcare JA. Exercise and multiple sclerosis. Medicine & Science in Sports & Exercise 1993;11(4):451???465.
Frau J, Coghe G, Lorefice L, Fenu G, Cadeddu B, Marrosu MG, et al. Attitude towards physical activity in patients with multiple sclerosis: a cohort study. Neurol Sci 2015 Jun;36(6):889-893.
Whittaker R. Issues in mHealth: findings from key informant interviews. J Med Internet Res 2012 Oct 02;14(5):e129
Jahns R, Houck P. Mobile Health Market Report 2013-2017. USA: Research 2 Guidance; 2013.
Marziniak M, Brichetto G, Feys P, Meyding-Lamadé U, Vernon K, Meuth SG. The Use of Digital and Remote Communication Technologies as a Tool for Multiple Sclerosis Management: Narrative Review. JMIR Rehabil Assist Technol 2018 Apr 24;5(1):e5
Lavorgna L, Brigo F, Moccia M, Leocani L, Lanzillo R, Clerico M, et al. e-Health and multiple sclerosis: An update. Mult Scler 2018 Nov 19;24(13):1657-1664.
Ancker JS, Witteman HO, Hafeez B, Provencher T, Van de Graaf M, Wei E. "You Get Reminded You're a Sick Person": Personal Data Tracking and Patients With Multiple Chronic Conditions. J Med Internet Res 2015 Aug 19;17(8):e202
Griffin N, Kehoe M. A questionnaire study to explore the views of people with multiple sclerosis of using smartphone technology for health care purposes. Disabil Rehabil 2018 Jun;40(12):1434-1442.
Giunti G, Kool J, Rivera Romero O, Dorronzoro Zubiete E. Exploring the Specific Needs of Persons with Multiple Sclerosis for mHealth Solutions for Physical Activity: Mixed-Methods Study. JMIR Mhealth Uhealth 2018 Feb 09;6(2):e37
Hamilton AD, Brady RRW. Medical professional involvement in smartphone 'apps' in dermatology. Br J Dermatol 2012 Jul;167(1):220-221.
Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P T 2014 May;39(5):356-364
O'Neill S, Brady RRW. Colorectal smartphone apps: opportunities and risks. Colorectal Dis 2012 Sep;14(9):e530-e534.
Wong SJ, Robertson GA, Connor KL, Brady RR, Wood AM. Smartphone apps for orthopaedic sports medicine – a smart move? BMC Sports Sci Med Rehabil 2015 Oct 12;7(1).
Connor K, Brady RRW, de Beaux A, Tulloh B. Contemporary hernia smartphone applications (apps). Hernia 2014 Aug;18(4):557-561.
Giunti G, Guisado-Fernandez E, Caulfield B. Connected Health in Multiple Sclerosis: A Mobile Applications Review. 2017 Presented at: IEEE 30th International Symposium on Computer-Based Medical Systems; June 2017; Thessaloniki, Greece p. 660-665.
Giunti G. 3MD for Chronic Conditions, a Model for Motivational mHealth Design: Embedded Case Study. JMIR Serious Games 2018 Aug 24;6(3):e11631
Ball MJ, Silva JS, Bierstock S, Douglas JV, Norcio AF, Chakraborty J, et al. Failure to Provide Clinicians Useful IT Systems: Opportunities to Leapfrog Current Technologies. Methods Inf Med 2018 Jan 18;47(01):4-7.
Hersh W. Health care information technology: progress and barriers. JAMA 2004 Nov 10;292(18):2273-2274.
Giunti G, Guisado-Fernandez E, Belani H, Lacalle-Remigio JR. Mapping the Access of Future Doctors to Health Information Technologies Training in the European Union: Cross-Sectional Descriptive Study. J Med Internet Res 2019 Aug 12;21(8):e14086
Giunti G, Guisado Fernández E, Dorronzoro Zubiete E, Rivera Romero O. Supply and Demand in mHealth Apps for Persons With Multiple Sclerosis: Systematic Search in App Stores and Scoping Literature Review. JMIR Mhealth Uhealth 2018 May 23;6(5):e10512
Part 210: Human-centred design for interactive systems. In: Ergonomics of human-system interaction. Geneva, Switzerland: International Organization for Standardization; 2010.
De Vito Dabbs A, Myers BA, Mc Curry KR, Dunbar-Jacob J, Hawkins RP, Begey A, et al. User-centered design and interactive health technologies for patients. Comput Inform Nurs 2009;27(3):175-183
Pruitt J, Adlin T. The Persona Lifecycle: Keeping People in Mind Throughout Product Design. San Francisco, CA: Morgan Kaufmann Publishers Inc; 2005.
Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform 2005 Feb;38(1):75-87
Rogers Y, Sharp H, Preece J. Interaction Design: Beyond Human - Computer Interaction. Hoboken, NJ: Wiley; 2011.
Lathan CE, Sebrechts MM, Newman DJ, Doarn CR. Heuristic evaluation of a web-based interface for internet telemedicine. Telemed J 1999 Jul;5(2):177-185.
Jaspers MWM. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform 2009 May;78(5):340-353.
Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, et al. A user-centered model for designing consumer mobile health (mHealth) applications (apps). J Biomed Inform 2016 Apr;60:243-251
Kinzie M, Cohn WF, Julian MF, Knaus WA. A user-centered model for web site design: needs assessment, user interface design, and rapid prototyping. J Am Med Inform Assoc 2002;9(4):320-330
Krupp LB, LaRocca NG, Muir-Nash J, Steinberg AD. The fatigue severity scale. Application to patients with multiple sclerosis and systemic lupus erythematosus. Arch Neurol 1989 Oct 01;46(10):1121-1123.
Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessely S, Wright D, et al. Development of a fatigue scale. Journal of Psychosomatic Research 1993 Feb;37(2):147-153.
Giunti G. Gamified Design for Health Workshop. Stud Health Technol Inform 2016;225:605-606.
Giunti G, Mylonopoulou V, Rivera Romero O. More Stamina, a Gamified mHealth Solution for Persons with Multiple Sclerosis: Research Through Design. JMIR Mhealth Uhealth 2018 Mar 02;6(3):e51
Chen ES, Cimino JJ. Automated discovery of patient-specific clinician information needs using clinical information system log files. AMIA Annu Symp Proc 2003:145-149
Thompson AJ, Banwell BL, Barkhof F, Carroll WM, Coetzee T, Comi G, et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. The Lancet Neurology 2018 Feb;17(2):162-173.
Vaughn J, Summers-Goeckerman E, Shaw RJ, Shah N. A Protocol to Assess Feasibility, Acceptability, and Usability of Mobile Technology for Symptom Management in Pediatric Transplant Patients. Nursing Research 2019;68(4):317-323.
Kurtzke JF. Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS). Neurology 1983 Nov 01;33(11):1444-1444.
Cella D, Lai JS, Nowinski CJ, Victorson D, Peterman A, Miller D, et al. Neuro-QOL: Brief measures of health-related quality of life for clinical research in neurology. Neurology 2012 May 09;78(23):1860-1867.
Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. J Med Internet Res 2006 Nov 14;8(4):e27
Georgsson M, Staggers N. Quantifying usability: an evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. J Am Med Inform Assoc 2016 Jan;23(1):5-11
Jaspers MWM, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform 2004 Nov;73(11-12):781-795.
Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004 Feb;37(1):56-76
Nielsen J. Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI conference on Human Factors in Computing Systems. New York, NY: Association for Computing Machinery; 1994 Presented at: SIGCHI conference on Human Factors in Computing Systems; April 1994; Boston, MA p. 152-158.
Nielsen J. Finding usability problems through heuristic evaluation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: Association for Computing Machinery; 1992 Presented at: SIGCHI Conference on Human Factors in Computing Systems; 1992; Monterey, California p. 373-380.
Sandelowski M. Whatever happened to qualitative description? Res. Nurs. Health 2000 Aug;23(4):334-340.
Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006 Jan;3(2):77-101.
Shenton AK. Strategies for Ensuring Trustworthiness in Qualitative Research Projects. Education for Information 200 Jul;22(2):63-75.
Leur KD, Drukker J, Christiaans H, Rijk TD. Cultural differences in product design: a study of differences between the South Korean and the Dutch kitchen environment. JDR 2006;5(1):16.
Evers V, Day D. The Role of Culture in Interface Acceptance. In: Human-Computer Interaction INTERACT 97. Boston, MA: Springer; 1997.
Lee K, Harada A. Cultural effects on subjective preference. Cross-Cultural Study between Korea and Japan. KANSEI Engineering International 2000;1(2):51-60.
Perri S, Shao A, Swai N, Mitchell M, Staggers N. Crucial issues in think aloud techniques for cross cultural studies. Stud Health Technol Inform 2014;205:863-867.
Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union. Europe: The European Parliament and The Council of the European Union; 2016. URL: http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L:2016:119:TOC [accessed 2020-07-19]
Ethical Principles of Research in the Humanities and Social and Behavioural Sciences and Proposals for Ethical Review. Finland: National Advisory Board on Research Ethics; 2009. URL: http://www.tenk.fi/sites/tenk.fi/files/ethicalprinciples.pdf [accessed 2020-07-19]
Código de buenas prácticas en investigación universidad de sevilla. Seville, Spain: Universidad de Sevilla; 2017:1-17.
Tur C. Fatigue Management in Multiple Sclerosis. Curr Treat Options Neurol 2016 Jun 18;18(6):26
Hourihan SJ. Managing fatigue in adults with multiple sclerosis. Nurs Stand 2015 Jun 24;29(43):51-58.
© 2020. This work is licensed under https://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Rivera-Romero, Octavio
; Kool, Jan
; Bansi, Jens
; Sevillano, Jose Luis
; Granja-Dominguez, Anabel
; Izquierdo-Ayuso, Guillermo
; Giunta, Diego