Reliability and Repeatability of the Assessment of Stress in Nursing Students Instrument in Podiatry Students: A Transcultural Adaptation

Abstract

Background: This study aimed to adapt the Spanish version of the Assessment of Stress in Nursing Students (ASNS) (shorter form) instrument for Spanish podiatry students, labeling the new tool as the Spanish version of Assessment of Stress in Podiatry Students (ASPS). The adaptation for reliability and repeatability included performing the transcultural adaptation process and examining the repeatability and reliability of ASPS when used in a different language. Methods: An internationally recommended translation procedure was used to adapt transculturally this tool. The instrument’s test–retest reliability was evaluated in two sessions that were 10 days apart. Results: After considering each domain’s total score, the reliability and internal consistency were analyzed with Cronbach’s α and intraclass correlation coefficient for a 95% confidence interval. Good internal consistency was reported according to total score (α = 0.8626) and each one of the six domains: (1) the practical activities performance domain showed a Cronbach’s α of 0.8684; (2) professional communication domain, α = 0.8765; (3) time management domain, α = 0.8832; (4) environment domain, α = 0.8974; (5) professional education domain, α =0.873; and (6) theoretical activity domain, α = 0.8787. Test–retest reliability, by paired test of Wilcoxon, was not significant, showing that there were not differences between domain scores (p ≥ 0.05). Lastly, visual distributions of Bland and Altman plots did not provide differences between domains and total scores. Conclusions: The Spanish version of the ASPS showed good repeatability, reliability and acceptability to measure stress parameters for podiatry students.

1. Introduction

Lazarus and Folkman (1986) defined stress as, “A particular relationship between the individual and the environment that is evaluated by that one as threatening or overflowing of its resources and that puts endangering their well-being.” Stress has different meanings and can develop as a result of different types of situations and relationships, such as work, family and academics [1].

New university students, especially those in health sciences programs, report high levels of stress with negative health consequences related with academic performance and wellbeing [2,3].

First, this results from the fact that they are facing a new, totally unknown environment that is different from their secondary education environment, which forces behavioral changes in order to adapt to the new setting [4]. Second, the responsibility that comes with dealing with patients and their pathologies is extremely stressful [5].

One of the factors most clearly linked to the appearance of stress in new students is associated with work overload [6]. It is necessary to differentiate between an excess of work (quantitative overload) and the excessive difficulty and complexity of it (qualitative overload). As a consequence of the limited amount of time that can be dedicated to their studies, students’ personal limitations, a curriculum with an excessive number of subjects and/or excessive demands, in addition to other reasons, students can become overwhelmed [7].

Determination of the negative impacts related to the best options to fight stress during the university years and posterior working life, may improve the overall health and the mental wellbeing of people, and their sanitary work health [8].

In the context of podiatric studies, there are no reliability and repeatability measurement instruments to quantify stress in students. We cannot find many studies that measured stress in podiatry students using general scales for measuring stress, such as the Undergraduate Sources of Stress Survey (USOS) [6,7,8,9], or test–retest reliability tools or ad hoc qualitative scales [10].

One reliability and repeatability instrument which showed adequate psychometric properties was the tool named Assessment of Stress in Nursing Students (ASNS) [11]. This tool comprised 30 items and six domains: (1) practical activity performance domain; (2) professional communication domain; (3) time management domain; (4) environment domain; (5) professional education domain; and (6) theoretical activity domain. The factor analysis of this tool’s psychometric properties showed that the instrument is reliable, resulting in Cronbach’s alpha (α) values that ranged from 0.71–0.87 [12].

This tool was previously used to determine stress parameters in nursing students, leading to excellent findings [12,13,14].

This instrument has appropriate domains for measuring stress in the podiatric context; thus, this study aimed to perform the transcultural adaptation of the nursing items to the podiatry context to create the Spanish version of the tool named Assessment of Stress in Podiatry Students (ASPS) and test its reliability and repeatability [15,16].

2. Methods 2.1. ASPS Study Design

A cross-sectional descriptive design was used following the declaration for strengthening the reporting of observational studies in epidemiology (STROBE) statement and verification of its checklist for study items. The transcultural process, reliability and repeatability assessments were performed by the ASNS that was modified for podiatry students. The exclusion criteria included several parameters: (1) previous history of work in a medical setting—i.e., nursing assistant or other kinds of hospital work occupations, such working at day care centers or similar settings; (2) refusal to sign the informed consent; (3) inability to understand the tool instructions to fill out the test (this exclusion criterion was self-reported by the study’s participants) and/or to carry them out; and (4) not being of Spanish nationality [17].

We followed the Declaration of Helsinki and human rights and biomedicine declarations. We contacted the author of the original questionnaire (ASNS) in order to gather information about this research, the questionnaire’s translation and its subsequent reliability and repeatability. Second, forward translation was performed by two independent bilingual Spanish translators. Third, reconciliation of the forward translations was performed by each translator separately. Fourth, the reconciled forward translation and item modification of the questionnaire’s podiatry context was translated back into the original language by two authors, both of whom are podiatrists and university professors with PhDs. Fifth, the translated version was compared with the original version in order to confirm the conceptual equivalence of the translation and to identify any discrepancies or unclear terms. Sixth, in in order to reach an agreement about the translation, the harmonization was carried out by an expert panel comprised of six authors: five podiatrists and a physiotherapist, all of whom are university professors with PhDs. Seventh, cognitive interviews were carried out at three podiatry schools in order to establish the instrument’s reliability and repeatability and avoid potential errors [18,19,20,21].

Lastly, a proofread version regarding ASPS comprised the same number of items (i.e., 30 items) with the same Likert scale which was applied for the original version. 2.2. Test–Retest Reliability Study and Sample Size Calculation

The test–retest reliability was tested during 2 sessions through these links: https://docs.google.com/forms/d/e/1FAIpQLSeUcE8BZDL1KLrOJMQ_jV9pYp51-5MVJzYmsMIDYW0cIf_xTw/viewform?c=0&w=1; https://docs.google.com/forms/d/e/1FAIpQLSfPnHFmuW_l1qrxxpYei341crV0PBdTjWUWGfRBlqTuu435SQ/viewform?c=0&w=1.

The students completed the questionnaires using a 10 days lapse between both two sessions. In addition to filling out ASPS items, socio-demographic data, including gender and age, were collected in that session. Two hundred participants were recruited via a consecutive sampling method, of whom 183 were enrolled into the study from the freshman classes of the University Complutense of Madrid, University of Extremadura and Universidade da Coruña (Spain). Thus, students presented similar socio-demographic characteristics. Study participants were developing their clinical podiatric practices at these universities. A pilot study was first conducted in order the establish participants’ linguistic understanding of the ASPS. The minimum sample size was calculated using the point biserial model correlation with the G*Power 3.1.9.2 software. Indeed, a 2-tailed hypothesis, a moderate effect size of 0.4, an α error probability of 0.01 with IC 99% and a β error of 0.10 with statistical power (1-β error probability) of 0.90 were used for the sample size calculation. The total calculated sample size was 82 subjects. Finally, a total of 183 students were included in this study.

The recommended sample size for testing test–retest reliability is at least 100 (Kline, 1999) [22].

2.3. Statistical Analysis

All of the variables were examined in order to determine the normality of the distribution using the Kolmogorov-Smirnov test, and the data were considered normally distributed if p > 0.05. Independent Student’s t-tests were performed in order to determine whether there were statistically significant differences in cases in which normal distributions were seen. Regarding the total and each one of the domain scores, the instrument’s internal consistency was analyzed by α for a 95% confidence interval (95% CI). Our α values were interpreted according to the following internationally accepted steps: (1) 0.5–0.59 = no value of test; (2) 0.6–0.69 = poor value; (3) 0.7–0.79 = moderate value; (4) 0.8–0.89 = good value; and (5) 0.9–1 = excellent value [23].

For the statistical analysis of reliability over time, the intraclass correlation coefficient (ICC) two-way mixed effect, absolute agreement single rater/measurement was measured. Furthermore, the Wilcoxon paired test or and paired t student test was used to determine the presence of systematic differences between scores of test-retest for no normal and normal distribution of variables, respectively. For interpreting the values of ICC, benchmarks values according to Landis and Koch were followed [17]: (1) ≤0.20, slight agreement; (2) 0.21 to 0.4, fair; (3) 0.41 to 0.6, moderate; (4) 0.61 to 0.8, substantial; and (5) ≥0.81, almost perfect. Furthermore, standard errors of measurement (SEM) were calculated to measure the range of error for each item according the formula SEMs = SD × sqrt (1−ICC). SEM is a quantitative expression for the range of error that can occur whenever the same participant repeats certain tests [24,25].

Coefficients of variation (CVs) were calculated for the absolute comparison of parameters. The CVs were calculated for intrasession reliability. The CV was calculated as the mean normalized to the SD. This value represents the amount of variation between trials, normalized to the mean for each variable. A higher CV value shows greater heterogeneity of variable values. The 95% limits of agreement (LoA) between sessions and devices expressed the degrees of error proportional to the means of the measurement units, and these statistics were calculated using the methods described by Bland and Altman, and if the differences between the measurements tended to agree, the results were close to zero. Minimal detectable changes (MDCs) were calculated in order to determine the minimal changes’ magnitudes to get a 95% CI regarding changes observed between two tests to reflect the true changes and avoid measurement errors, and were calculated according to SEMs values (SEMS = 1.96 × SEM × √2).

Finally, repeatability and heteroscedasticity were determined using the method of Bland and Altman plots [25] to display the agreement between test and retest. These plots showed the difference between each pair of measurements on the y-axis against the mean of each pair of measurements on the x-axis.

A p-value < 0.01 with a confidence interval of 99% was considered statistically significant for all tests (SPSS for Windows, version 20.0; SPSS Inc., Chicago, IL, USA). 2.4. Ethics Approval and Consent to Participate Hospital Universitario Clínico San Carlos Ethics Committee (Spain), with number 18-278-E. All those who volunteered to take part in the study gave their informed written consent. 2.5. Availability of Data and Supporting Materials Section The dataset supporting the conclusions of this article is available from [email protected] in the Research, Health and Podiatry Unit. Department of Health Sciences. Faculty of Nursing and Podiatry. Universidade da Coruña, Spain. 3. Results 3.1. Translation A non-randomized and consecutive sampling method was used to select the 200 participants, of whom 183 gave consent and were enrolled into the study.

The final sample of students who filled out the two questionnaires comprised 183 subjects (140 women and 43 men). The students’ mean age was 23.59 ± 5.19 years old (range: 18–52). The forward translations procedures were carried out with one discrepancy between the two versions. Due to the comments of some of the students with respect to pre-test item 25 (“Vivir la experiencia, como podólogo/a en formación, que me ofrecen las actividades de las prácticas clínicas”), the item was re-formulated for the post-test in order to make it more understandable (“Vivir la experiencia de ser un podólogo/a en formación, y de lo que me ofrecen las actividades de las prácticas clínicas”). All of the participants stated that the questionnaire questions were very important for the evaluation of self-perceived stress levels in podiatric clinical practices. Finally, none of the participants stated that the questionnaire was excessively long or boring, and the estimated average time to complete it was no more than 5 min (see Appendix A).

3.2. Reliability and Repeatability All variables showed a normal distribution (p < 0.05) except for total test and total retest (p > 0.05).

The α Cronbach shown in Table 1 for total scores demonstrated good post-test internal consistency with an α = 0.883. In addition, very good internal consistency was determined in all questionnaire domains: (1) practical activities performance domain showed α = 0.868; (2) professional communication domain, α= 0.876; (3) time management domain, α = 0.883; (4) environment domain, α = 0.897; (5) professional education domain, α = 0.873; and (6) finally, theoretical activity domain, α = 0.883.

Table 1 also shows the questionnaire’s reliability and reproducibility results and systematic differences across the 30 items and six domains, and there were no systematic differences in any of items, including total scores from the questionnaires (p > 0.01)

The ICC shows values that varied from 0.89 to 0.976. showing almost perfect test–retest reliability for all items and total score. Test–retest agreement assessed by SEMs, CVs and LoA was very low, showing that differences between the measurements tended to agree due to the results being close to zero, ranging from 0 to 0.019, 0 to 0.17 and −0.09 to 0.17, respectively, showing substantial agreement. The LoA (95% CI) of the measurements between the test and retest showed values for all dimensions which tended to have almost perfect agreement, showing no variability. The MDC ranged from 0 to 0.057, showing absolute accuracy.

In addition, visual distribution of the Bland and Altman graphs comparing both total test and retest scores for individual participants (Figure 1) and plots for six domain test retest scores (Figure 2a–f) for individual participants did not show heteroscedasticity or systematic errors of measurements.

4. Discussion

The adapted ASPS for measuring stress factors among podiatry students showed good Cronbach’s alpha coefficients and ICCs; for this reason and according to the guidelines for the process of cross–cultural adaptation of self–reported measures [22] and following the recommendations for evaluating Cronbach’s alpha coefficients, the ASPS is a test retest reliable tool.

The ASPS showed good reliability and repeatability with adequate internal consistency to evaluate stress in podiatry students, and the questionnaire also shows excellent reproducibility; all of the p–values from the Wilcoxon paired test were not statistically significant in any dimension. In addition, the Bland and Altman method shows whether there are differences between the tests (pre and post-tests) in order to check the reproducibility of the test results and whether the differences are systematic or random. After analyzing our results, we can conclude that there are no differences between the pre and post-test total scores or domain scores.

Our results are comparable to the original test retest reliability of Costa and Polak’s (2009) questionnaire due to higher Cronbach’s alpha values being found according to total and domain scores and all domains, and also better ICCs by each domain and item [18]. Our sample was similar to that of Costa and Polak (2009) with respect to age, sex and number of participants [18].

We were not able to compare our reliability data for the total items (α = 0.8626) and domains (performance of practical activities: α = 0.8684, professional communication domain: α = 0.8765, time management domain: α = 0.8832, environment domain: α = 0.8974, professional education domain: α = 0.873 and theoretical activity domain: α = 0.8787) with other tools that are used to measure stress in podiatry students because there is no specific and directly–oriented instrument for these students at this time. For this reason, we compared our results with prior scientific studies in which general instruments were used to assess stress in podiatry or chiropody students.

We found better values with the ASPS when compared with the USOS questionnaire. This tool was used in a study by Mandy et al. (2013) with podiatry students from the United Kingdom and Australia [14]. The USOS questionnaire has 18 items in three subscales (academic demands α =0.85, financial issues α = 0.8 and personal issues α = 0.73) [11]. Similar values were found after using the 10–item perceived stress scale (α = 0.84) [26,27]. This scale was used in a study by Al Sawah et al. (2015) with first and second–year podiatry students that aimed to investigate the effects of coffee intake, energy drink consumption and perceived stress on sleep quality in these students during their preclinical studies [28,29].

The Sisco Inventory of Academic Stress [29] is composed of 31 items with an α coefficient of 0.90, and the academic stressors scale [7,30,31] is composed of a total of 54 items with a Cronbach’s α coefficient that oscillates between 0.79 and 0.96. The α values of these instruments were very similar to those found for the ASPS, but the items and dimensions are not focused on the podiatry setting alone.

One of the study’s limitations was not considering age and sex due to the sample size and greater number of females compared to males. Additionally, a larger and more diverse (individuals from other universities) sample size would help to improve the strength of the study. Finally, a questionnaire can only gather agree/disagree data, but not the opinions behind the answers, and further investigation is needed in podiatry students to study this important problem. 5. Conclusions The adapted ASPS for measuring stress factors among podiatry students did not lose its test–retest reliability upon translation, and the substitutions of terms and adjustments from the original version demonstrated good psychometrics properties; thus, it is a reliable tool to be used in the Spanish population and may be also used in future studies.

Figure 1. Bland-Altman plot comparing total test and retest scores for individual participants using the Assessment of Stress in Podiatry Students (ASPS).

Figure 2. Bland and Altman plots for six domain test retest scores for individual participants according to the ASPS.

Figure 2. Bland and Altman plots for six domain test retest scores for individual participants according to the ASPS.

Figure 2. Bland and Altman plots for six domain test retest scores for individual participants according to the ASPS.

Item/Dimesion	Test N = 183			Retest N = 183			Test Retest Reliability
	Mean ± SD (CI 95%)	Item–Total Correlation (p-Value)	Alpha Cronbach If Item Removed	Mean ± SD (CI 95%)	Item–Total Correlation (p-Value)	Alpha Cronbach If Item Removed	p-Value	ICC (IC 95%)	SEM	MDC	CV	LoA (IC 95%)
Item 5	1.95 ± 0.89 (1.82–2.08)	0.529 (<0.01)	0.864	1.86 ± 0.89 (1.73–1.99)	0.509 (<0.01)	0.883	0.018 *	0.914 (0.884–0.936)	0.018	0.050	2.3	−0.09 (−1.06/0.88)
Item 7	1.58 ± 0.93 (1.45–1.72)	0.563 (<0.01)	0.863	1.60 ± 0.89 (1.60–1.73)	0.574 (<0.01)	0.882	0.569 *	0.913 (0.883–0.935)	0.003	0.010	0.52	0.00 (−1.00/1.00)
Item 9	2.45 ± 0.72 (2.35–2.56)	0.453 (<0.01)	0.869	2.46 ± 0.68 (2.35–2.5)	0.460 (<0.01)	0.886	0.712 *	0.890 (0.852–0.918)	0.000	0.000	0.00	0.00 (−0.87/0.87)
Item 21	2.47 ± 0.74 (2.36–2.58)	0.442 (<0.01)	0.826	2.42 ± 0.76 (2.51–2.53)	0.476 (<0.01)	0.847	0.117 *	0.907 (0.875–0.930)	0.011	0.030	1.01	−0.05 (−0.91/0.81)
Item 4	1.53 ± 0.85 (1.40–1.65)	0.499 (<0.01)	0.865	1.63 ± 0.84 (1.40–1.65)	0.527 (<0.01)	0.881	0.937 *	0.976 (0.834–0.908)	0.000	0.000	0.00	0.00 (−1.10/1.10)
Item 12	1.83 ± 0.98 (1.69–1.97)	0.601 (<0.01)	0.822	1.79 ± 0.93 (1.65–1.92)	0.584 (<0.01)	0.845	0.100 *	0.947 (0.929–0.960)	0.007	0.020	1.21	–0.04 (−0.89/0.80)
Dimesion Performance of Practical Activities	1.97 ± 0.57 (1.89–2.05)	0.778 (<0.01)	0.849	1.94 ± 0.55 (1.86–2.02)	0.792 (<0.01)	0.868	0.094 *	0.948 (0.930–0.961)	0.004	0.012	0.70	0.03 (−0.46/0.52)
Item 6	1.14 ± 0.89 (1.01–1.27)	0.483 (<0.01)	0.866	1.21 ± 0.87 (1.09–1.34)	0.534 (<0.01)	0.881	0.059 *	0.875 (0.831–0.906)	0.019	0.053	3.26	0.10 (−1.10/1.20)
Item 8	1.06 ± 0.89 (0.92–1.19)	0.414 (<0.01)	0.869	1.10 ± 0.92 (0.97–1.24)	0.479 (<0.01)	0.883	0.290 *	0.901 (0.867–0.926)	0.011	0.030	2.28	0.00 (−1.00/1.10)
Item 16	1.41 ± 0.96 (1.27–1.55)	0.480 (<0.01)	0.829	1.45 ± 0.93 (1.31–1.59)	0.489 (<0.01)	0.850	0.265 *	0.925 (0.899–0.944)	0.009	0.024	1.53	0.04 (−0.94/1.03)
Item 20	1.60 ± 1.01 (1.45–1.75)	0.355 (<0.01)	0.833	1.59 ± 0.97 (1.45–1.60)	0.300 (<0.01)	0.856	0.949 *	0.884 (0.845–0.913)	0.001	0.004	0.17	–0.00 (−1.30/1.30)
Dimesion: Proffessional Communication	1.30 ± 0.67 (1.20–1.40)	0.606 (<0.01)	0.859	1.34 ± 0.65 (1.25–1.44)	0.637 (<0.01)	0.877	0.152 *	0.908 (0.876–0.931)	0.009	0.024	1.55	–0.04 (−0.79/0.71)
Item 18	1.65 ± 0.98 (1.51–1.80)	0.480 (<0.01)	0.827	1.62 ± 0.93 (1.48–1.75)	0.498 (<0.01)	0.856	0.295 *	0.929 (0.904–0.947)	0.007	0.020	1.17	–0.04 (−1.01/0.94)
Item 3	1.57 ± 1.03 (1.42–1.72)	0.394 (<0.01)	0.876	1.56 ± 0.98 (1.41–1.70)	0.431 (<0.01)	0.891	0.752 *	0.949 (0.932–0.962)	0.002	0.005	0.35	–0.01 (−0.88/0.86)
Item 26	1.65 ± 0.94 (0.00–3.00)	0.528 (<0.01)	0.823	1.62 ± 0.94 (1.48–1.75)	0.586 (<0.01)	0.846	0.258 *	0.936 (0.914–0.952)	0.007	0.019	1.17	–0.04 (−0.95/0.97)
Item 30	1.81 ± 0.95 (1.67–1.95)	0.494 (<0.01)	0.823	1.81 ± 0.95 (1.67–1.05)	0.552 (<0.01)	0.845	0.969 *	0.942 (0.922–0.957)	0.000	0.000	0.00	0.00 (−0.87/0.87)
Item 23	1.43 ± 0.81 (1.32–1.55)	0.298 (<0.01)	0.832	1.48 ± 0.81 (1.36–1.60)	0.392 (<0.01)	0.852	0.275 *	0.881 (0.840–0.911)	0.011	0.030	1.50	0.04 (−1.00/1.69)
Dimesion: Time Management	1.62 ± 0.67 (1.53–1.72)	0.779 (<0.01)	0.871	1.61 ± 0.67 (1.52–1.71)	0.672 (<0.01)	0.883	0.796 *	0.955 (0.939–0.966)	0.001	0.004	0.27	0.01 (–0.54/0.56)
Item 11	1.10 ± 0.99 (0.96–1.25)	0.390 (<0.01)	0.829	1.09 ± 0.99 (0.94–1.23)	0.402 (<0.01)	0.886	0.608 *	0.955 (0.940–0.966)	0.002	0.007	0.75	−0.02 (−0.83/0.79)
Item 29	0.93 ± 1.03 (0.78–1.08)	0.443 (<0.01)	0.826	0.95 ± 1.01 (0.80–1.10)	0.419 (<0.01)	0.849	0.432 *	0.965 (0.953–0.974)	0.003	0.008	1.16	0.02 (−0.72/0.76)
Item 22	0.94 ± 1.04 (0.79–1.09)	0.452 (<0.01)	0.825	0.95 ± 1.00 (0.80–1.10)	0.433 (<0.01)	0.848	0.678 *	0.959 (0.945–0.970)	0.002	0.004	0.58	0.01 (−0.79/0.81)
Item 24	1.14 ± 1.01 (0.99–1.29)	0.414 (<0.01)	0.827	1.15 ± 1.01 (1.00–1.30)	0.425 (<0.01)	0.849	0.655 *	0.941 (0.921–0.958)	0.002	0.005	0.48	0.00 (−0.90/1.00)
Dimesion: Environment	1.03 ± 0.92 (0.89–1.16)	0.473 (<0.01)	0.881	1.03 ± 0.91 (0.90–1.17)	0.462 (<0.01)	0.897	0.917 *	0.939 (0.920–0.954)	0.001	0.002	0.33	−0.01 (−0.63/0.62)
Item 1	2.46 ± 0.69 (2.36–2.56)	0.259 (<0.01)	0.835	2.52 ± 0.64 (2.43–2.62)	0.341 (<0.01)	0.853	0.050 *	0.896 (0.860–0.922)	0.014	0.038	1.21	0.06 (−0.74/0.86)
Item 15	1.93 ± 0.83 (1.81–2.06)	0.607 (<0.01)	0.820	1.91 ± 0.82 (1.79–2.03)	0.561 (<0.01)	0.845	0.545 *	0.903 (0.870–0.928)	0.005	0.013	0.57	−0.02 (−0.99/0.94)
Item 17	2.15 ± 0.80 (2.03–2.27)	0.484 (<0.01)	0.827	2.12 ± 0.77 (2.00–2.23)	0.477 (<0.01)	0.849	0.321 *	0.920 (0.892–0.940)	0.007	0.018	0.77	−0.03 (−0.88/0.81)
Item 19	1.93 ± 0.86 (1.80–2.06)	0.522 (<0.01)	0.825	1.92 ± 0.82 (1.80–2.04)	0.576 (<0.01)	0.844	0.993 *	0.896 (0.861–0.922)	0.001	0.003	0.14	–0.00 (−1.00/1.00)
Item 25	1.45 ± 0.84 (1.32–1.57)	0.667 (<0.01)	0.818	1.53 ± 0.86 (1.41–1.66)	0.667 (<0.01)	0.842	0.021 *	0.887 (0.849–0.916)	0.021	0.057	2.94	0.10 (−1.00/1.10)
Item 27	2.13 ± 0.75 (2.02–2.24)	0.506 (<0.01)	0.823	2.11 ± 0.74 (2.000–2.22)	0.530 (<0.01)	0.845	0.486 *	0.904 (0.871–0.928)	0.005	0.013	0.52	−0.02 (−0.90/0.85)
Dimesion: Professional Education	2.01 ± 0.53 (1.93–2.09)	0.778 (<0.01)	0.854	2.02 ± 0.53 (1.94–2.10)	0.776 (<0.01)	0.873	0.579 *	0.938 (0.916–0.953)	0.002	0.005	0.27	0.67 (−0.63/1.96)
Item 10	2.46 ± 0.67 (2.36–2.56)	0.287 (<0.01)	0.831	2.43 ± 0.71 (2.32–2.53)	0.400 (<0.01)	0.851	0.304 *	0.892 (0.856–0.920)	0.008	0.021	0.67	–0.03 (−0.88/0.81)
Item 13	2.54 ± 0.69 (2.44–2.64)	0.445 (<0.01)	0.825	2.49 ± 0.72 (2.38–2.60)	0.493 (<0.01)	0.846	0.050 *	0.939 (0.919–0.955)	0.009	0.024	0.98	–0.06 (−0.71/0.61)
Item 14	1.83 ± 0.77 (1.72–1.94)	0.381 (<0.01)	0.829	1.81 ± 0.77 (1.70–1.93)	0.518 (<0.01)	0.847	0.600 *	0.850 (0.800–0.888)	0.005	0.013	0.45	–0.09 (−1.10/1.10)
Item 2	2.04 ± 0.80 (1.92–2.16)	0.357 (<0.01)	0.830	2.01 ± 0.80 (1.89–2.12)	0.411 (<0.01)	0.849	0.246 *	0.914 (0.885–0.936)	0.008	0.022	0.95
Item 28	2.03 ± 0.75 (1.92–2.14)	0.490 (<0.01)	0.823	3.00 ± 0.75 (1.88–2.11)	0.562 (<0.01)	0.879	0.306 *	0.911 (0.881–0.934)	0.007	0.019	0.82	−0.03 (−0.88/0.81)
Dimesion: Theoretical Activity	2.18 ± 0.47 (2.11–2.25)	0.616 (<0.01)	0.867	2.15 ± 0.50 (2.07–2.22)	0.718 (<0.01)	0.883	0.138 *	0.923 (0.896–0.942)	0.007	0.019	0.79	0.03 (−0.48/0.54)
Total	1.74 ± 0.40 (1.68–1.80)	1.00	0.842	1.74 ± 0.40 (1.68–1.80)	1.00	0.883	0.723 **	0.957 (0.943–0.968)	0.001	0.002	0.12	0.00 (−0.32/0.33)

Note: Item 25, original term “nursing student” has been changed to “podiatry student.” Abbreviations: SD, standard deviation; CI 95%, confidence interval 95%; NA, not applicable; ICC, intraclass correlation index; N/A, not applicable. * p-value from Wilcoxon signed-rank test; ** p-value from paired Student t-test; SEM, standard error of measurement; CV, coefficient of variation; SEM%, percent error of the SEM; MDC = minimum detectable change; LoA, limits of agreement. A p-value < 0.01 with a confidence interval of 99% was considered statistically significant.

Author Contributions

All authors: concept, design, analyses, interpretation of data and drafting of manuscript or revising it critically for important intellectual content. Conceptualization, P.P.-L., M.E.L.-I., R.B.-d.-B.-V., D.L.-L., D.R.-S., M.S.Á.-G., J.M. and C.C.-L.; Data curation, P.P.-L.; Formal analysis, P.P.-L., M.E.L.-I., R.B.-d.-B.-V., D.L.-L., M.S.Á.-G. and C.C.-L.; Investigation, P.P.-L., M.E.L.-I., R.B.-d.-B.-V., D.L.-L., D.R.-S., M.S.Á.-G., J.M. and C.C.-L.; Methodology, P.P.-L., M.E.L.-I., R.B.-d.-B.-V., D.L.-L., D.R.-S., M.S.Á.-G., J.M. and C.C.-L.; Supervision, M.E.L.-I., R.B.-d.-B.-V., D.L.-L., D.R.-S., M.S.Á.-G., J.M. and C.C.-L.; Validation, P.P.-L., M.E.L.-I., R.B.-d.-B.-V., D.R.-S., M.S.Á.-G., J.M. and C.C.-L.; Visualization, D.L.-L. and M.S.Á.-G.; Writing - original draft, P.P.-L., M.E.L.-I., R.B.-d.-B.-V., D.L.-L., D.R.-S., M.S.Á.-G., J.M. and C.C.-L.; Writing - review & editing, P.P.-L., M.E.L.-I., R.B.-d.-B.-V., D.L.-L., D.R.-S., M.S.Á.-G., J.M. and C.C.-L. All authors have read and agreed to the published version of the manuscript.

Funding

The sponsors had no role in the design, methods, subject recruitment, data collection, analysis or preparation of paper; and the authors had no personal relationships with other people or organizations that could inappropriately influence (bias) their work.

Acknowledgments

To all students who participated in the research.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

ASPS	Assessment of Stress in Podiatry Students
SD	Standard deviation

Appendix A. Assessment of Stress in Podiatry Students

Likert Scale	0 = I do not live the situation; 1 = I don't feel stressed with the situation; 2 = I feel little stress with the situation; 3 = I feel very stressed with the situation
Item 1	Concerns about professional future.
Item 2	Obligation to do extra class assignments.
Item 3	Reduced social interactions because of feelings of loneliness.
Item 4	Performing the general assistance procedures.
Item 5	New situations one may experience in clinical practice.
Item 6	Communication with the other professionals at the training place.
Item 7	Environment of the training clinical practice.
Item 8	Communication with professionals of other sectors at the training place.
Item 9	Fear of making mistakes while assisting patients.
Item 10	Format of method used to assess theoretical content.
Item 11	Reduced social interactions because of feelings of loneliness.
Item 12	Performing certain assistance procedures.
Item 13	Feelings of insecurity or fear while taking theoretical exam.
Item 14	Level of difficulty of extra class assignments
Item 15	Similarities between situations experienced during the training process and those that may be experienced during professional life.
Item 16	Perception of difficulties regarding relationships with other professionals.
Item 17	Thinking of situations that may be experienced during professional life.
Item 18	Little time to spend with family members.
Item 19	Perceptions of professional responsibility while doing the training program.
Item 20	Identification of contradictory attitudes in the other professionals.
Item 21	Feeling of not having enough knowledge for the practical test.
Item 22	Public transportation used to go to school.
Item 23	Time demanded by the professor to prepare extra class activities.
Item 24	Distance between most training places and residences.
Item 25	Experiencing activities in the training field as a podiatry student.
Item 26	Lack of leisure time.
Item 27	Perceptions of the theoretical knowledge acquired during the course.
Item 28	Understanding the theoretical and practical content taught in the classroom.
Item 29	Public transportation used to go to the training place.
Item 30	Lack of time to rest.
Item 31	Control Question A. How much time have you invested in completing this questionnaire? In minutes
Item 32	Control Question B. Was too long for you?
Item 33	Control Question C. Does everything you ask seem of interest to you about the self-perception of your stress level when practicing at the University Clinic?
Item 34	Control Question D. Are there items in which you have problems or doubts to interpret what is asked?
Item 35	Control Question E. Are there any questions that you cannot assess accurately enough or have doubts in the answer? Indicate which one/s:
Item 36	Control Question G. Is there a term or concept that you do not understand or have doubts about how to interpret it? Please indicate it:

References

1. Folkman, S.; Lazarus, R.S.; Gruen, R.J.; DeLongis, A. Appraisal, coping, health status, and psychological symptoms. J. Pers. Soc. Psychol. 1986, 50, 571-579.

2. Jayasankara Reddy, K.; Rajan Menon, K.; Thattil, A. Academic Stress and its Sources Among University Students. Biomed. Pharmacol. J. 2018, 11, 531-537.

3. Tomaschewski-Barlem, J.G.; Lunardi, V.L.; Lunardi, G.L.; Barlem, E.L.; Silveira, R.S.; Vidal, D.A. Burnout syndrome among undergraduate nursing students at a public university. Rev. Lat. Am. Enfermagem. 2014, 22, 934-941.

4. Fonseca, J.R.; Calache, A.L.; Santos, M.R.; Silva, R.M.; Moretto, S.A. Association of stress factors and depressive symptoms with the academic performance of nursing students. Rev. Esc. Enferm. USP 2019, 53, 03530.

5. Olvera Alvarez, H.A.; Provencio-Vasquez, E.; Slavich, G.M.; Laurent, J.G.C.; Browning, M.; McKee-Lopez, G.; Robbins, L.; Spengler, J.D. Stress and Health in Nursing Students: The Nurse Engagement and Wellness Study. Nurs. Res. 2019, 68, 453-463.

6. Michie, F.; Glachan, M.; Bray, D. An Evaluation of Factors Influencing the Academic Self-concept, Self-esteem and Academic Stress for Direct and Re-entry Students in Higher Education. Educ. Psychol. 2001, 21, 455-472.

7. Palomo-López, P.; Becerro-de-Bengoa-Vallejo, R.; Calvo-Lobo, C.; Tovaruela-Carrión, N.; Rodríguez-Sanz, D.; Elena Losa-Iglesias, M.; López-López, D. Student perceptions of the education environment in a Spanish medical podiatry school. J. Foot Ankle Res. 2018, 11, 14.

8. Cabanach, R.G.; Souto-Gestal, A.; Franco, V. Escala de Estresores Académicos para la evaluación de los estresores académicos en estudiantes universitarios. Rev. Iberoam. Psicol. Salud 2016, 7, 41-50.

9. Van der Riet, P.; Rossiter, R.; Kirby, D.; Dluzewska, T.; Harmon, C. Piloting a stress management and mindfulness program for undergraduate nursing students: Student feedback and lessons learned. Nurse Educ Today. 2015, 35, 44-49.

10. Tinley, P. Occupational stress among Australian podiatric physicians in general and geriatric practice. J. Am. Podiatr. Med. Assoc. 2015, 105, 130-134.

11. Hogan, V.; Coggins, M.A.; Roberts, N.; Kelly, M. Health and safety in podiatric medicine: Findings from a national survey of Irish podiatric physicians. J. Am. Podiatr. Med. Assoc. 2017, 107, 522-530.

12. Magnavita, N.; Chiorri, C. Academic stress and active learning of nursing students: A cross-sectional study. Nurse Educ. Today 2018, 68, 128-133.

13. Blackmore, A.; Tucker, B.; Jones, S. Development of the Undergraduate Sources of Stress Questionnaire. Int. J. Ther. Rehabil. 2005, 12, 99-105.

14. Mandy, A.; Tucker, B.; Tinley, P. Sources of stress in undergraduate podiatry students in the UK and Australia. Int. J. Ther. Rehabil. 2006, 13, 109-117.

15. Jacob, T.; Itzchak, E.B.; Raz, O. Stress among healthcare students-A cross disciplinary perspective. Physiother. Theory Pract. 2013, 29, 401-412.

16. Weir, E.C.; Carline, T.E. Short Communication: Reactions of first-year podiatry students to cadaver dissection. Med. Teach. 1997, 19, 311-313.

17. Losa-Iglesias, M.E.; Jiménez-Fernández, R.; Alameda-Cuesta, A.; Cid-Exposito, M.G.; Rodriguez-Vazquez, R.; Becerro-De-Bengoa-Vallejo, R. Reliability and repeatability of the instrument for the assessment of stress in nursing students (ASNS). Medicina 2019, 55, 634.

18. Costa, A.L.S.; Polak, C. Construção e validação de instrumento para Avaliação de Estresse em Estudantes de Enfermagem (AEEE). Rev. da Esc. Enferm. da USP 2019, 55, 634.

19. Hirata Soares, M.; Santana Oliveira, F. The relation between alcohol, tobacco and stress in nursing students marcos. SMAD. Rev. eletrônica saúde Ment. álcool e Drog. 2013, 9, 88-94.

20. Benavente, S.B.; Silva, R.M.; Higashi, A.B.; Guido, L.D.; Costa, A.L. Influence of stress factors and socio-demographic characteristics on the sleep quality of nursing students. Rev. Esc. Enferm. USP 2014, 48, 514-520.

21. Beaton, D.E.; Bombardier, C.; Guillemin, F.; Ferraz, M.B. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila. Pa. 1976) 2000, 25, 3186-3191.

22. Wild, D.; Grove, A.; Martin, M.; Eremenco, S.; McElroy, S.; Verjee-Lorenz, A.; Erikson, P. ISPOR Task Force for Translation and Cultural Adaptation Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: Report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Heal. 2005, 8, 94-104.

23. Kline, P. Handbook of Psychological Testing, 2nd ed.; Taylor and Francis: New York, NY, USA, 1999.

24. Swets, J.A. Measuring the accuracy of diagnostic systems. Science 1988, 240, 1285-1293.

25. Landis, J.R.; Koch, G.G. The measurement of observer agreement for categorical data. Biometrics 1977, 33, 159-174.

26. Stratford, P.W.; Goldsmith, C.H. Use of the standard error as a reliability index of interest: An applied example using elbow flexor strength data. Phys. Ther. 1997, 77, 745-750.

27. Bland, J.M.; Altman, D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet (London UK) 1986, 1, 307-310.

28. Cohen, S.; Kamarck, T.; Mermelstein, R. A Global Measure of Perceived Stress. J. Health Soc. Behav. 1983, 24, 385.

29. Sawah, M.A.; Ruffin, N.; Rimawi, M.; Concerto, C.; Aguglia, E.; Chusid, E.; Infortuna, C.; Battaglia, F. Perceived Stress and Coffee and Energy Drink Consumption Predict Poor Sleep Quality in Podiatric Medical Students. J. Am. Podiatr. Med. Assoc. 2015, 105, 429-434.

30. Macías, B. Arturo Un modelo conceptual para el estudio del estrés académico. Rev. Electrónica Psicol. Iztacala. 2006, 9, 110-129.

31. Escorcia, M.E.; Osorio, H.; Perez, C.A.; Thulcan, M.C.; Caballero-Uribe, C.V.; Borda-Perez, M. Efectividad de una Intervención Educativa en Pacientes con Artritis Reumatoide. Rev Col Reum. 2006, 13, 11-19.