Development and psychometric properties of a short form of the Illness Denial Questionnaire

Abstract

Background: Coping with chronic illness can be overwhelming for patients and caregivers, and may be inhibited by the denial mechanism, and therefore, denial represents a critical issue for health professionals. Assessing illness denial is far from easy, and brief tools suitable for medical settings are lacking. In this paper, the development of a short form of the Illness Denial Questionnaire (IDQ) for patients and caregivers is presented.

Methods: In study 1, the IDQ was administered to 118 patients and 83 caregivers to examine the internal structure of denial; then the properties of the items (DIF, fit, and difficulty) were evaluated according to the Rasch model in order to select the best items for the Illness Denial Questionnaire-Short Form (IDQ-SF). Study 2 included 202 participants (113 patients and 89 caregivers). The internal structure of the IDQ-SF was tested via confirmatory factor analysis (CFA). Reliability and concurrent validity were also studied using the Anxiety and Depression Questionnaire-Reduced Form (AD-R).

Results: The CFA showed a two-factor structure encompassing “Denial of negative emotions” and “Resistance to change”. Results of the Rasch analyses led to the selection of 4 items for each dimension. The resulting IDQ-SF (8 items) showed a two-factor structure as well as good reliability and concurrent validity with AD-R.

Conclusion: The IDQ-SF represents a valid tool for quickly evaluating the core of illness denial in patients and caregivers. This brief and easily administrable questionnaire allows health professionals to outline the presence and severity of illness denial in order to set individually tailored interventions.

Introduction

In recent decades, increasing attention has been paid in medical settings to clinically relevant psychological aspects given their well-known key role in influencing the medical care process in negative and positive ways.¹ Among the several psychological issues involved in medical settings, illness denial is one of the most controversial due to the scarce clarity in its definition and the consequent difficulties in its assessment.^2,3 Denial was firstly introduced by Sigmund Freud as an unintentional defense mechanism that can be triggered by a distressful and threatening situation.^4,5

Concerning medical settings, denial might be triggered by the onset of an illness and may consist of the negation of some aspects related to it.^6,7 Indeed, denial can affect different areas, from the diagnosis itself up to subjective emotions and feelings, behavioral lifestyle changes,^8,9 and modifications of interpersonal relationships.¹⁰ Furthermore, illness denial can affect patients and their caregivers, who are frequently strongly involved in the process of physically and psychologically caring for their loved ones.¹¹

According to Freud, denial represents an initial phase in the process of achieving awareness of something that is repressed, without necessarily accepting it.⁵ Thus, the cognitive-affective elaboration process of the illness might encompass fluctuant phases through different intensities of denial, which may finally result in awareness but not always in acceptance.¹² Even if denial can be considered a justified reaction in some phases of the illness, its persistency may be dysfunctional as it suppresses the adaptation process to reality.¹³ An excessive use of denial may indicate that it would be harder to accept the reality of illness.¹⁴ For example, dysfunctional denial may delay help-seeking requests for serious illness symptoms, thus postponing treatments,¹⁵ or it could imply poor compliance with treatment or even involve an inadequate attitude of the caregiver in supporting the patient.^16,17 Furthermore, denial needs to be distinguished from conscious, aware, and deliberate avoidance of the threatening condition.¹⁸

It is important for health professionals to assess illness denial with reliable tools, but there is difficulty in retrieving specific and useful tools for this purpose – as underlined in previous studies.¹⁸ Indeed, the literature reports that semi-structured interviews¹⁹ or self-report questionnaires²⁰ have addressed only specific disorders and often have assessed something ambiguous instead of proper denial.

The Illness Denial Questionnaire (IDQ)¹⁸ was created with the aim to assess denial in relation to illness, in patients and in caregivers and independently of the specific kind of illness or disturbance involved. The IDQ is a self-report questionnaire consisting of 24 dichotomous items (true/false) that show good psychometric properties and a three-factor structure. The IDQ assesses three dimensions: "Denial of negative emotions," "Resistance to change," and "Conscious avoidance." "Denial of negative emotions" refers to the emotional reactivity, a subjective dimension, related to the individual's emotional life and to its regulation. This dimension encompasses different emotions and feelings, such as sadness, insecurity, worry, and anger.²¹ "Resistance to change" concerns the behavioral efforts necessary to manage illness, and it is aimed at assessing the practical changes and modifications in one's lifestyle in order to adapt to a new situation (eg, medications, therapies). Such transformation and reorganization mainly concern individual behaviors and habits.²² "Denial of negative emotions" and "Resistance to change" have been found to be the core components of denial.¹⁸ Besides these, even a third dimension is present, namely "Conscious avoidance." This corresponds to a later and less severe critical stage of the illness elaboration process in which awareness is reached. At this point, acceptance of the illness's existence can begin.²³ Despite the validity of the conscious avoidance construct, it could be confounding to measure it together with denial since they are well-distinguished concepts: denial implies a lack of awareness and is a preliminary phase characterized by the removal of unpleasant material from consciousness; whereas in conscious avoidance, awareness is present but the individual voluntarily avoids facing a threatening situation.^23,24 Furthermore, "Conscious avoidance" has proven to be independent and different from the core aspects of illness denial – both theoretically and psychometrically.¹⁸ Illness denial represents a more severe clinical issue than conscious avoidance, and it is also more difficult to assess given the lack of awareness. Thus, denial represents a critical issue for clinical practice, and the present work is specifically aimed to assess it. Given this premise, the present study is focused on the central aspects of illness denial: "Denial of negative emotions" and "Resistance to change." Despite their interconnection, they concern two well-distinguished areas of the individual's life: emotional and behavioral.

Medical settings require short and reliable tools given their undoubted advantages: easy administration, suitable integration in longer routine assessment batteries, suitable administration to subjects with reduced attention ability, and rapid administration in medical wards such as waiting rooms.^25,26 Brief tools can be used for monitoring the individual's progress over time, thus informing clinical decisions. The IDQ requires a time-consuming administration and a long scoring procedure, thus making it difficult for health professionals to immediately understand the actual denial condition of the patient/caregiver. To our knowledge, nowadays, brief tools specifically measuring illness denial are lacking. For these reasons, a short form of the IDQ is needed in order to conduct an efficient and valid assessment of illness denial. Furthermore, since denial affects patients and caregivers, a unique assessment tool suitable for both is desirable.

This paper is aimed at developing and validating a brief version of the IDQ, which will be called Illness Denial Questionnaire–Short Form (IDQ-SF). In Study 1, the factor structure of the IDQ considering the two dimensions of "Denial of negative emotions" and "Resistance to change" was tested via confirmatory factor analysis (CFA). Then, the properties of the items were evaluated with the Rasch model analysis²⁷ in order to select the items with the best psychometric properties for the IDQ-SF. In Study 2, the factor structure and psychometric properties of the IDQ-SF were investigated.

Study 1 Materials and methods

Participants In line with the previous study,¹⁸ participants were enrolled in two rehabilitation hospitals of IRCCS Maugeri Scientific Institutes in Veruno and Pavia, Italy. The sample comprised 201 participants (56% females) with an overall mean age of 56.14 years (SD=12.80). Among the participants, 118 were inpatients (45% females; for females: mean age=56.08, SD=12.42; for males: mean age=58.72, SD=10.20). The remaining 83 subjects were caregivers (71.08% females; for females: mean age=53.17, SD=14.62; for males: mean age=56.63 years, SD=14.37). Caregivers displayed different kinds of relationship with patients: 47 spouses/living-with (56.6%), 1 partner/not-living-with (1.2%), 6 parents (7.2%), 21 sons/daughters (25.3%), and 8 in other types of relationships (9.6%). The participants, patients and caregivers, were retrieved from several medical wards: cardiology (26.37%), neurology (24.88%), oncology (22.40%), nephrology (12.94%), and pneumology (7.5%). More information about patients and caregivers is reported in Table 1.

Instruments

The IDQ¹⁸ was administered to participants who were asked to express in a dichotomous form (false=0; true=1) if their experience was represented by each of the 24 statements. The IDQ evaluates three dimensions: "Denial of negative of emotions" (7 items; eg, "This disorder/disease frightens me"), "Resistance to change" (9 items; eg, "Nothing in my life will change on account of this disorder/disease"), and "Conscious avoidance" (8 items; eg, "I try not to pay any attention to my disorder/disease"). The first two dimensions were found to express the real form of denial, while the latter dimension of "Conscious avoidance" represented a more advanced stage of the illness elaboration process.¹⁸ After recoding the reverse items, higher scores on the IDQ expressed higher levels of denial. A psychologist introduced the study. All participants gave informed consent to use their data in anonymous form. The ethical committee of the Maugeri Scientific Institute discussed and approved the study.

Statistical analyses

In the first part of Study 1, the IDQs factor structure was tested through CFA, and reliability and validity were evaluated. Only the two main factors of denial were considered: "Denial of negative emotions" and "Resistance to change."¹⁸ The diagonally weighted least squares (DWLS) estimator was used²⁸ given its suitability for categorical data. The goodness-of-fit was evaluated considering chi square, the χ²/df ratio, the comparative fit index (CFI),²⁹ the Tucker–Lewis index (TLI),³⁰ the root mean square of approximation (RMSEA),³¹ and the adjusted goodness-of-fit statistic (AGFI).³² A non-significant chi square expresses adequate model fit. However, chi-square is sensible to sample-size.³³ The χ²/df ratio is a handy measure of fit,^34–36 with values smaller than 3 indicating good fit.³⁷ AGFI estimates the part of variance accounted for by the estimated population covariance;³⁸ values greater than 0.90 denote well-fitting models. The CFI and the TLI express the amount of variance and covariance accounted for by the model compared with an unstructured baseline model. These indexes are unaffected by sample size.^39,40 Desirable CFI and TLI values are above 0.95, although values between 0.90 and 0.95 are acceptable.²⁹ The RMSEA describes fit per degrees of freedom of the model; values between 0.05 and 0.08 denote acceptable fit³⁹ and values lower than 0.05 express good fit.⁴¹ Reliability was evaluated for each of the two dimensions using Cronbach's α.⁴² Values≥0.7 suggest acceptable-to-high scale reliability.

Furthermore, in order to create the reduced form of the questionnaire, a Rasch model analysis was used to select the items with the best psychometric properties.^27,43,44 There is a long history of applications of Rasch models in the medical field.^45–48 Given the specific measurement properties of Rasch models, they have been widely used for the development of brief measurement.^49–53 Rasch models characterize the responses of persons to items as a function of person and item measures. These measures pertain to the level of a quantitative latent trait possessed by a person or item, and their specific meaning relies on the subject of the assessment. In educational assessments, for instance, person measures indicate the ability of persons, and item measures indicate the difficulty of items. In health status assessments, person measures indicate the health of persons, and item measures indicate the severity of items. In the context of IDQs, providing a "true" answer to an item is analogous to stating the presence of the measured latent trait (ie, denial). The Rasch model is usually applied to evaluate psychometric tests since it is centered on the study of items and their properties. Items are ordered according to their severity (location or beta): items with less "true" responses are more severe, and items with more "true" responses are less severe. Severity is represented on a scale with mean zero – the higher the values, the higher the severity. Items should cover all the distinct levels of the trait of interest in order to properly measure it. Items covering the same level of the latent construct are redundant and can thus be dropped. An adequate coverage of the severity continuum suggests content representativeness and construct validity.⁵⁴ Fit indices show how well the data fit the model, and their technical quality is evaluated. Item fit addresses the extent to which an item is related to the rest of the scale. Unfitting items may be considered for elimination. A valid measurement tool should cover different levels of the trait continuum and should not contain items with differential item functioning⁵⁵ (DIF). In the presence of DIF, comparisons among groups are questionable: they should only be qualitative and not quantitative.⁴⁵ The absence of DIF items allows quantitative comparisons among scores of different groups to be made.⁵⁴

Item reduction strategy Given that unidimensionality is an assumption of the Rasch model, the two dimensions of denial were separately analyzed by fitting two Rasch models, one for "Denial of negative emotions" and the other for "Resistance to change." Given the equal distribution of items across the components, the same number of items was selected for each dimension. The length of the short form was not a priori defined but was suggested by statistical analyses. A minimum of 4 items per dimension was set in order not to weaken the scale too much. A two-step procedure was applied in order to create the short form: a) poorly functioning items were dropped; b) then, only the best items were selected for the short-form.

First of all, DIF was analyzed to test the invariance of each item across different groups of subjects (patients and caregivers) and across sex (females and males). An item exhibiting DIF is differently answered by subjects with the same ability level but from different groups, and such nuisance factors affect goodness of measurement.⁵⁶ DIF results were considered in the selection of items since it is preferable to drop items showing DIF with large effect size⁵⁵ in order to obtain a test suitable for different subjects across sex and across groups (ie, patients and caregivers). Uniform and nonuniform DIF were considered: DIF is uniform if the item–group interaction is independent of the subjects' ability level, while DIF is nonuniform when the item functioning is related to the subjects' ability level.⁵⁷

Then, Rasch fit indices were used to evaluate item properties. Good fit indices mean that a certain item fits to the Rasch model's expectations based on item difficulties and subjects' ability level. Infit and outfit are sensitive to unexpected responses given a certain person's trait level. Infit is more precise and detects unexpected responses to items that are close to a person's trait level, while outfit perceives "bigger" unexpected responses to items that are far from the subject's trait level. Fit indices rely on the mean square of the standardized residuals for items, which are not sample-size dependent. The expected value for fit indices is 1, but recommended values should not exceed 1.4 and might not be lower than 0.7.⁵⁸ Values above 1.4 suggest that there is unexplained variance (ie, underfit), while values lower than 0.7 mean that there is redundancy among the items (ie, overfit). Items showing outfit values over 1.4 were dropped, and items with fit indices closer to 1 were selected to be included in the short form.

Furthermore, the item severity (beta) was considered in order to select the items that allowed for covering the entire continuum, thus providing a well-graded tool assessing different levels of the measured dimension without redundant items. Finally, the clinical meaning of each item was critically discussed by expert clinicians and was then considered to wisely drive the item choice in order to assess all the relevant clinical areas.

All the data analyses were performed with R statistical environment (v. 3.5.0).⁵⁹ CFA was performed by using the "lavaan" package (v. 0.6.3),⁶⁰ while Rasch analyses were conducted with TAM (v. 2.12.18)⁶¹ and "difR" (v. 5.0)⁵⁷ packages.

Results

Results from the CFA on the two dimensions of denial showed a two-factor structure with good fit indices. Despite the chi-square fit reaching statistically significant values [χ²=208.747; df=103; p<0.001], the other fit indices showed satisfying results. The CFI and the TLI values were above the desirable threshold, suggesting good fit (CFI=0.960; TLI=0.954). The RMSEA suggested acceptable fit [RMSEA=0.072 (90% CI from 0.058 to 0.086); p (RMSEA<0.05)=0.007]. The χ²/df ratio of the model⁶² suggested the acceptability of the model (χ²/df=2.026; <3).³⁵ AGFI further suggested a well-fitting model (AGFI=0.911). As shown in Table 2, all the items significantly loaded on the intended factor (p<0.001; mean_loadings =0.662; SD_loadings =0.154), ranging from 0.338 (Item12) to 0.928 (Item#6). Furthermore, a positive correlation between the two factors was found (r=0.619; p<0.001). In addition, both dimensions of denial showed good internal consistency (α=0.76 for "Denial of negative emotions"; α=0.72 for "Resistance to change").

Two Rasch models were separately fitted for the two dimensions of denial. Table 3 reports the parameters considered in the selection process. After fitting the Rasch model on the "Denial of negative emotions" items, in the first step of selection all critical items showing DIF were dropped: Item#2 and Item#17 showed DIF between patients and caregivers; Item#10 showed DIF across sex. In the second step, the remaining 4 items showed acceptable infit and outfit values: none exceeded 1.4 or was under the lower threshold. Those items well covered the latent trait continuum, and their clinical content was considered relevant and meaningful. Thus, the items for the short form were: Item#6, Item#7, Item#21, and Item#23. Given these results, the length for each scale was settled at 4 items.

A second Rasch model was fitted on the "Resistance to change" items. In the first step of selection, no item showed DIF across sex, and only one item (Item#4) showed critical DIF across patients and caregivers and thus was dropped. In the second step, all the 8 remaining items showed satisfactory Rasch fit indices. Among them, the items with fit indices closer to the expected value of 1 were chosen. Also, the difficulty (beta parameter) and the clinical connotation were considered in the selection. Item#12 was chosen given its specific clinical meaning. The items in the couples Item#5–Item#13 and Item#11–Item#16 showed similar severity, but given the clinical meaning and the fit values, the items Item#5 and Item#16 were preferred. Finally, items Item#15 and Item#22 showed the same severity, but Item#22 was preferred given its clinical meaning. Thus, the other 4 items selected for the short form were Item#5, Item#12, Item#16, and Item#22. The resulting IDQ-SF is a brief tool aimed at assessing illness denial. It is composed of 8 items, 4 from "Denial of negative emotions" (Item#6, Item#7, Item#21, Item#23) and 4 from "Resistance to change" (Item#5, Item#12, Item#16, Item#22).

Study 2

Study 2 was aimed at testing in a new sample the validity and the psychometric properties of the IDQ-SF developed in Study 1.

Materials and methods

Participants This sample comprised 202 participants (58% females; mean age=55.93, SD=13.88). The inpatients group counted 113 subjects (47% females; for females: mean age=58.60, SD=13.00; for males: mean age=59, SD=12.93). The caregiver group was composed of 89 subjects (71.91% females; for females: mean age=52.02, SD=13.36; for males: mean age=52.88, SD=16.68). Caregivers showed different relationships with patients: 49 spouses/living-with (55.1%); 1 partner/not-living-with (1.1%); 4 parents (4.5%); 23 sons/daughters (25.8%); and 12 in another kind of relationship (13.5%). Patients and caregivers were enrolled from different medical wards: oncology (26.73%), neurology (24.26%), cardiology (23.27%), nephrology (16.34%), and pneumology (3.96%). Descriptive statistics are reported in Table 1. After a short presentation made by a psychologist, all participants signed the informed consent allowing their data to be used in anonymous form. The study was discussed and approved by the ethical committee of the Maugeri Scientific Institute.

Instruments

Participants were presented with the IDQ-SF – developed in Study 1. They were asked to answer the 8 dichotomous items of the questionnaire with true or false according to their experience.

The Anxiety and Depression Questionnaire–Reduced Form (AD-R) was used for evaluating state anxiety and depression symptoms.⁶³ It is a reduction of the State-Trait Anxiety Inventory⁶⁴ and the Depression Questionnaire.⁶⁵ The anxiety subscale measures state anxiety with 10 items on a 4-level-rating scale (not at all=1, somewhat=2, moderately so=3, and very much so=4). The total score ranges between 0 and 40. The depression subscale measures depression symptoms with 15 dichotomous items (no=0; yes=1). The total score of this scale ranges from 0 to 15.

Statistical analyses

The factorial structure of IDQ-SF was studied via CFA. Given the results from Study 1 and from previous literature,¹⁸ a two-factor structure was hypothesized. The DWLS was used as an estimator, and several indicators were considered to evaluate model fit: χ², χ²/df, CFI, TLI, RMSEA, and AGFI. Internal consistency was evaluated using Cronbach's α. Concurrent validity was studied by correlating the IDQ-SF with the anxiety and depression subscales of the AD-R questionnaire.

Results

The CFA on the IDQ-SF tested the two-factor model where 4 items saturated in the factor "Denial of negative emotions" and 4 items saturated in the factor "Resistance to change." Fit indices showed that the two-factor structure of the IDQ-SF provided a good depiction of the item responses: χ² (19)=28.408; p=0.076. The χ²/df ratio of the model suggested the acceptability of the model (χ²/df=1.49; <3). Fit indices suggested ideal fit (CFI=0.987; TLI=0.981). The RMSEA further suggested good fit [RMSEA =0.50 (90% CI: from 0.000 to 0.086); p (RMSEA<0.05)=0.466]. AGFI further suggested a well-fitting model (AGFI=0.937). Each item significantly loaded on the intended factor [mean_loadings=0.717; SD_loadings=0.099, ranging from 0.604 to 0.86 (all ps<0.001)]. The factors showed a positive moderate correlation of 0.500. The standardized coefficients are presented in Table 4. Internal consistency was good for "Denial of negative emotions" (α=0.71) and for "Resistance to change" (α=0.58).

Concerning concurrent validity, correlations were calculated separately for patients and caregivers. The IDQ-SF total score showed a negative moderate correlation with anxiety subscales (patients=–0.57; caregivers=–0.45) and depression subscales (patients=–0.48; caregivers=–0.39). More information is provided in Table 5.

General discussion

The present paper aimed at developing and validating a short form of the IDQ for patients and caregivers, considering the core dimensions of "Denial of negative emotions" and "Resistance to change." Study 1 was aimed at developing the short form. Results of this study confirmed the validity and reliability of the IDQ, as composed of these two related but different dimensions. According to hypotheses and in line with the previous study, results of CFA confirmed that these dimensions represent two factors concerning different aspects of the individual's life: one on the behavioral level and the other on the emotional level. Results of CFA were an introductory phase for the item reduction in which a Rasch model was independently fitted on the items of each of the two denial dimensions.

Rasch models represent valuable tools for the development and validation of brief assessment instruments. After considering the results of several statistics, Rasch analysis led to the selection of 4 items of "Denial of negative emotions" and 4 items of "Resistance to change," thus leading to the IDQ-SF, a reduced version consisting of 8 items that were invariant across patients and caregivers. Thanks to the Rasch model properties, the items of the IDQ-SF are presented on a growing level of denial, thus allowing clinicians to immediately visualize if denial is present and its severity. If Rasch had not been used, poorly functioning items could have been included in the short form.

Study 2 aimed at testing the validity of the short-form in a new sample: the factor structure of IDQ-SF was confirmed by a second CFA. The IDQ-SF also showed good psychometric properties, and, as found in the original form, it displayed negative correlations with anxiety and depression symptoms. This result makes sense from a theoretical point of view: emotions, such as anxiety and depression, are suppressed by the denial defense mechanism.^18,66

The IDQ-SF may have useful applications in the clinical field. By considering item severity, the answers of a subject can be evaluated to provide meaningful information to clinicians and health professionals. The number and the severity of endorsed items could indicate the psychological risk of a person and the urgency of clinical intervention.

This study is not free of limitations, and its results should be considered object of discussion by future studies. Our sample mainly involved patients affected by chronic diseases in the rehabilitation setting and their caregivers. Future research should consider acute phases of illness and post-rehabilitation conditions.^67–70 An interesting future research topic would be to investigate the relationship among illness denial and other psychological conditions, such as well-being, stress, and caregivers' strain. Further studies are needed to extend or disconfirm our results and to deepen further aspects of the denial and illness elaboration process in order to improve the efficacy and quality of the offered clinical treatment.

Conclusion

Illness denial represents a critical issue in clinical practice, not always evenly defined and thus difficult to assess. In a previous study, denial was defined as a process in which one's perspective may fluctuate during the course of the illness before its acceptation.

The IDQ-SF represents a brief tool focusing on denial's core dimensions referring to the emotional experience and behavioral lifestyle change as previously emerged. Given its handy structure, the IDQ-SF's administration is fast, fitting for patients with low attention span, and also suitable for integration in longer assessment batteries or to monitor the individual's progress over time. Its interpretation and scoring are easier for clinicians, allowing an immediate depiction of the patient's/caregiver's intensity of denial, thus favoring the appropriate clinical intervention. In fact, psychological interventions need to promote the illness elaboration process, keeping in account that it fluctuates from denial to awareness to acceptance. The negative correlation among denial, anxiety, and depression⁶⁶ confirms that the individuals are not ready to cope with the illness in an effort to protect themselves from the perceived threat: this suggests a cautious approach by the clinicians in order to avoid even worse reactions and to help along the elaboration process of the illness.

Even if more studies are needed to deepen the role of denial in the illness elaboration process, the IDQ-SF may be a starting point for its assessment and in catalyzing the clinical and scientific discussion about this intriguing topic.

Abbreviations

IDQ, Illness Denial Questionnaire; IDQ-SF, Illness Denial Questionnaire-Short Form; AD-R, Anxiety and Depression Questionnaire-Reduced Form; DIF, Differential Item Functioning.

Ethics approval and informed consent

The present manuscript includes two studies on human subjects. Informed consent was retrieved by all participants, and both studies were approved by the Ethical Committee of Maugeri Scientific Institutes (Pavia); protocol number: 974CE.

Data availability

The data generated and analyzed in the current study are property of Maugeri Scientific Institutes S.p.A. Restrictions apply to the availability of these data, which were used under license for the current study and so are not publicly available.

Disclosure

The authors report no conflicts of interest in this work.

References

1. Eisner MD, Blanc PD, Yelin EH, et al. Influence of anxiety on health outcomes in COPD. Thorax. 2010;65(3):229–234. doi:10.1136/thx.2009.126201 2. Levitt EE. A note on "MMPI scale development methodology. J Pers Assess. 1978;42(5):503–504. doi:10.1207/s15327752jpa4205_10 3. McGrath RE, O'Malley WB. The assessment of denial and physical complaints: the validity of the Hy scale and associated MMPI signs. J Clin Psychol. 1986;42(5):754–760. doi:10.1002/1097-4679(198609)42:5<754::AID-JCLP2270420513>3.0.CO;2-O 4. Strachey J. The Ego and the Id: The Standard Edition of the Complete Psychological Works of Sigmund Freud. New York, NY: Norton & Company; 1960. doi:10.1177/105382590602900110 5. O'Neil MK, Akhtar S. On Freud's Negation. Karnac Books; 2011. Available from: https://books.google.it/books?hl=it‎&id=AA-37hl5X7EC&oi=fnd&pg=PR7&dq=o+neill+2011+on+freud+s+negation&ots=fIYXdMCTbj&sig=Wg03plaxR6-FOlUw29q8qPWy1G4&redir_esc=y#v=onepage&q&f=false.AccessedJanuary16, 2019. 6. Watson M, Greer S, Blake S, Shrapnell K. Reaction to a diagnosis of breast cancer relationship between denial, delay and rates of psychological morbidity. Cancer. 1984;53(9):2008–2012. doi:10.1002/1097-0142(19840501)53:9<2008::AID-CNCR2820530934>3.0.CO;2-B 7. Aitken-Swan J, Easson E. Reactions of cancer patients on being told their diagnosis. Br Med J. 1959; 1:779–783. doi: 10.1136/bmj.1.5124.779 8. Livneh H. Psychosocial adaptation to chronic illness and disability: a conceptual framework. Rehabil Couns Bull. 2001;44(3):151–160. doi:10.1177/003435520104400305 9. Livneh H, Antonak RF. Psychosocial adaptation to chronic illness and disability: a primer for counselors. J Couns Dev. 2005;83(1):12–20. doi:10.1002/j.1556-6678.2005.tb00575.x 10. Telford K, Kralik D, Koch T. Acceptance and denial: implications for people adapting to chronic illness: literature review. J Adv Nurs. 2006;55(4):457–464. doi:10.1111/j.1365-2648.2006.03942.x 11. Chiao CY, Wu HS, Hsiao CY. Caregiver burden for informal caregivers of patients with dementia: a systematic review. Int Nurs Rev. 2015;62(3):340–350. doi:10.1111/inr.12194 12. Afrell M, Biguet G, Rudebeck CE. Living with a body in pain - between acceptance and denial. Scand J Caring Sci. 2007;21(3):291–296. doi:10.1111/j.1471-6712.2007.00475.x 13. Stroebe M, Boelen PA, Van Den HM, Stroebe W, Salemink E. Ruminative coping as avoidance. Eur Arch Psychiatry Clin Neurosci. 2007;257(8):462–472. doi:10.1007/s00406-007-0746-y 14. Lazarus RS. Coping theory and research: past, present, and future. Psychosom Med. 1993;55:234–247. doi:10.1097/00006842-199305000-00002 15. Covino JM, Stern TW, Stern TA. Denial of cardiac illness: consequences and management. Prim Care Companion CNS Disord. 2011;13(5). doi:10.4088/PCC.11f01166 16. Rammohan A, Rao K. Burden and coping in caregivers of persons with schizophrenia. Indian J Psychiatry. 2002;44(3):220–227. 17. Ratti MM, Rossi A, Delli Zotti GB, Sarno L, Spotti D. Social support, psychological distress and depression in hemodialysis patients. Psicol Della Salut. 2017;(1):112–122. doi:10.3280/PDS2017-001006 18. Ferrario SR, Giorgi I, Baiardi P, et al. Illness denial questionnaire for patients and caregivers. Neuropsychiatr Dis Treat. 2017;13:909–916. doi:10.2147/NDT.S128622 19. Vos MS, Putter H, Leurs A, Rooijmans HGM, de Haes HCJM, van Houwelingen HC. The denial of cancer interview: development and first assessment of psychometric properties in lung cancer patients. Patient Educ Couns. 2007;67(1–2):224–234. doi:10.1016/j.pec.2007.03.019 20. Vanderdeycken W, Vanderlinden J. Denial of illness and the use of self‐reporting measures in anorexia nervosa patients. Int J Eat Disord. 1983;2(4):101–107. doi:10.1002/1098-108X(198322)2:4<101::AID-EAT2260020417>3.0.CO;2-U 21. Bowman GS. Emotions and illness. J Adv Nurs. 2001;34(2):256–263. doi:10.1046/j.1365-2648.2001.01752.x 22. Gignac MAM, Cott C, Badley EM. Adaptation to chronic illness and disability and its relationship to perceptions of independence and dependence. J Gerontol - Psychol Sci Soc Sci. 2000;55(6):362–372. doi:10.1093/geronb/55.6.P362 23. Lazarus RS. Fifty Years of the Research and Theory of R. S. Lazarus: An Analysis of Historical and Perennial Issues. London: Routledge Sources in History, Psychology Press; 1998. doi:10.1097/00001888-200009000-00007 24. Roth S, Cohen LJ. Approach, avoidance, and coping with stress. Am Psychol. 1986;41(7):813–819. 25. Paulhus DL, Vazire S. The self-report method. In: Robins RW, Krueger RC, editors. Handbook of Research Methods in Personality Psychology. New York: Guilford; 2005:224–239. doi:10.1016/B0-08-045044-X/00117-6 26. Vidotto G, Ferrario SR, Bond TG, Zotti AM. Family strain questionnaire - short form for nurses and general practitioners. J Clin Nurs. 2010;19:275–283. doi:10.1111/j.1365-2702.2009.02965.x 27. Rasch G. Probabilistic Models for Some Intelligence and Achievement Tests. Chicago: University of Chicago; 1980. doi:10.1016/S0019-9958(61)80061-2 28. Jöreskog KG, Sörbom DL. LISREL 7: User's Reference Guide. Chicago: Scientific Software International; 1989. 29. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107:238–246. doi:10.1037/0033-2909.107.2.238 30. Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1–10. doi:10.1007/BF02291170 31. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors.Testing Structural Equation Models. Newbury Park, CA: Sage Focus Editions; 1993:156. 32. Jöreskog K, Sörbom D. Lisrel VI Users' Guide. Mooresville (IL): Scientific Software; 1984. 33. Bentler PM, Bonett DG. Significance tests and goodness of fit in the analysis of covariance structures. Psychol Bull. 1980;88(3):588–606. doi:10.1037/0033-2909.88.3.588 34. Marsh HW, Hocevar D. Application of confirmatory factor analysis to the study of self-concept. first- and higher order factor models and their invariance across groups. Psychol Bull. 1985;97(3):562–582. doi:10.1037/0033-2909.97.3.562 35. Hoyle RH. Handbook of Structural Equation Modeling. New York: Guilford Press; 2012. doi:10.4135/9781412961288.n69 36. Pietrabissa G, Rossi A, Simpson S, et al. Evaluation of the reliability and validity of the Italian version of the schema mode inventory for eating disorders: short form for adult with dysfunctional eating behavior. Eat Weight Disord - Stud Anorexia, Bulim Obes. 2019:1–13. doi:10.1007/s40519-019-00644-5 37. Wheaton B, Muthén B, Alwin DF, Summers GF. Assessing reliability and stability in panel models. Sociol Methodol. 1977;8:84–136. doi:10.2307/270754 38. Tabachnick BG, Fidell LS. Using Multivariate Statistics.7thed. Boston: Pearson; 2007. doi:10.1037/022267 39. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model a Multidiscip J. 1999;6(1):1–55. doi:10.1080/10705519909540118 40. Hu L, Bentler PM. Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychol Methods. 1998;3(4):424–453. doi:10.1037/1082-989X.3.4.424 41. Browne MW, Cudeck R. Single sample cross-validation indices for covariance structures. Multivariate Behav Res. 1989;24(4):445–455. doi:10.1207/s15327906mbr2404_4 42. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334. doi:10.1007/BF02310555 43. Rasch G. Probabilistic models for some intelligence and achievement tests. Copenhagen Danish Inst Educ Res. 1960. doi:10.1016/S0019-9958(61)80061-2 44. Noventa S, Stefanutti L, Vidotto G. An analysis of item response theory and rasch models based on the most probable distribution method. Psychometrika. 2014;79(3):377–402. doi:10.1007/s11336-013-9348-y 45. Clauser BE, Ross LP, Nungester RJ, Clyman SG. An evaluation of the Rasch model for equating multiple forms of a performance assessment of physiciansʼ patient-management skills. Acad Med. 1997;72(Suppl. 1):76–78. doi:10.1097/00001888-199710001-00026 46. Fisher WP, Vial RH, Sanders CV. Removing rater effects from medical clerkship evaluations. Acad Med. 1997;72:443–444. doi:10.1097/00001888-199705000-00079 47. Da Dalt L, Anselmi P, Bressan S, et al. A short questionnaire to assess pediatric resident's competencies: the validation process. Ital J Pediatr. 2013;39:41. doi:10.1186/1824-7288-39-41 48. Da Dalt L, Anselmi P, Furlan S, et al. Validating a set of tools designed to assess the perceived quality of training of pediatric residency programs. Ital J Pediatr. 2015;41:2. doi:10.1186/s13052-014-0106-2 49. Vidotto G, Carone M, Jones PW, Salini S, Bertolotti G. Maugeri respiratory failure questionnaire reduced form: A method for improving the questionnaire using the Rasch model. Disabil Rehabil. 2007;29(13):991–998. doi:10.1080/09638280600926678 50. Vidotto G, Bertolotti G, Carone M, et al. A new questionnaire specifically designed for patients affected by chronic obstructive pulmonary disease. The Italian Health Status Questionnaire. Respir Med. 2006;100(5):862–870. doi:10.1016/j.rmed.2005.08.024 51. Colledani D, Robusto E, Anselmi P. Development of a new abbreviated form of the Junior Eysenck Personality Questionnaire-Revised. Pers Individ Dif. 2018;120:159–165. doi:10.1016/j.paid.2017.08.037 52. Colledani D, Anselmi P, Robusto E. Using item response theory for the development of a new short form of the eysenck personality questionnaire-revised. Front Psychol. 2018;9:1834. doi:10.3389/fpsyg.2018.01834 53. Colledani D, Anselmi P, Robusto E. Using multidimensional item response theory to develop an abbreviated form of the Italian version of Eysenck's IVE questionnaire. Pers Individ Dif. 2019;142:45–52. doi:10.1016/j.paid.2019.01.032 54. Smith EV. Evidence for the reliability of measures and validity of measure interpretation: a Rasch measurement perspective. J Appl Meas. 2001;2:281–311. 55. Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Human Sciences.3rded. New York: Routledge; 2015. doi:10.1207/S15327574IJT013&4_10 56. Ackerman TA. A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. J Educ Meas. 1992;29:67–91. doi:10.1111/j.1745-3984.1992.tb00368.x 57. Magis D, Béland S, Tuerlinckx F, de Boeck P. A general framework and an R package for the detection of dichotomous differential item functioning. Behav Res Methods. 2010;42(3):847–862. doi:10.1177/1359105315581516 58. Wright B, Linacre J, Gustafson J, Martin-Lof P. Reasonable mean-square fit values. Rasch Meas Trans. 1994;8:370. doi:10.1177/01461672012710004 59. R Core Team. R: a language and environment for statistical computing. Available from: http://www.R-project.org/. R Foundation for Statistical Computing, Vienna, Austria. doi:10.1007/978-3-540-74686-7 60. Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw. 2012;48(2):1–36. doi:10.18637/jss.v048.i02 61. Robitzsch A, Kiefer T, Wu M TAM: test analysis modules. R package version 2.12-18. 2018. doi:10.1177/0146621697211001 62. Jöreskog KG. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika. 1969;34(2):183–202. doi:10.1007/BF02289343 63. Moroni L, Bettinardi O, Vidotto G, et al. Scheda Ansia e Depressione forma ridotta: norme per l'utilizzo in ambito riabilitativo. Monaldi Arch Chest Dis. 2006;66(4):255–263. doi:10.4081/monaldi.2006.516 64. Spielberger CD, Gorsuch RL, Lushene RE. STAI manual for the state-trait anxiety inventory. Self-Evaluation Questionnaire. Manual. 1970;1–24. doi:10.1037/t06496-000 65. Bertolotti G, Michielin P, Vidotto G, Zotti AM, Sanavio E. Depression Questionnaire (DQ). In: Practitioner's Guide to Empirical Based Measures of Depression. Norwell (MA): Kluwer Academic/Plenum Pubblishers; 2000:45–47. 66. White KS, Pardue C, Ludbrook P, Sodhi S, Esmaeeli A, Cedars A. Cardiac Denial and psychological predictors of cardiac care adherence in adults with congenital heart disease. Behav Modif. 2016;40(1–2):29–50. doi:10.1177/0145445515613329 67. Kortte KB, Wegener ST. Denial of illness in medical rehabilitation populations: theory, research, and definition. Rehabil Psychol. 2004;49(3):187–199. doi:10.1037/0090-5550.49.3.187 68. Pietrabissa G, Manzoni GM, Rossi A, Castelnuovo G. The MOTIV-HEART study: a prospective, randomized, single-blind pilot study of brief strategic therapy and motivational interviewing among cardiac rehabilitation patients. Front Psychol. 2017;8:1–12. doi:10.3389/fpsyg.2017.00083 69. Jackson JB, Pietrabissa G, Rossi A, Manzoni GM, Castelnuovo G. Brief strategic therapy and cognitive behavioral therapy for women with binge eating disorder and comorbid obesity: a randomized clinical trial one-year follow-up. J Consult Clin Psychol. 2018;86(8):688–701. doi:10.1037/ccp0000313 70. Chan F, Da Silva Cardoso E, Chronister JA. Understanding Psychosocial Adjustment to Chronic Illness and Disability: A Handbook for Evidence-Based Practitioners in Rehabilitation. New York, NY: Springer Publishing Company; 2009. doi:10.1177/0034355210386240

Introduction In recent decades, increasing attention has been paid in medical settings to clinically relevant psychological aspects given their well-known key role in influencing the medical care process in negative and positive ways.¹ Among the several psychological issues involved in medical settings, illness denial is one of the most controversial due to the scarce clarity in its definition and the consequent difficulties in its assessment.^2,3 Denial was firstly introduced by Sigmund Freud as an unintentional defense mechanism that can be triggered by a distressful and threatening situation.^4,5 Concerning medical settings, denial might be triggered by the onset of an illness and may consist of the negation of some aspects related to it.^6,7 Indeed, denial can affect different areas, from the diagnosis itself up to subjective emotions and feelings, behavioral lifestyle changes,^8,9 and modifications of interpersonal relationships.¹⁰ Furthermore, illness denial can affect patients and their caregivers, who are frequently strongly involved in the process of physically and psychologically caring for their loved ones.¹¹ According to Freud, denial represents an initial phase in the process of achieving awareness of something that is repressed, without necessarily accepting it.⁵ Thus, the cognitive-affective elaboration process of the illness might encompass fluctuant phases through different intensities of denial, which may finally result in awareness but not always in acceptance.¹² Even if denial can be considered a justified reaction in some phases of the illness, its persistency may be dysfunctional as it suppresses the adaptation process to reality.¹³ An excessive use of denial may indicate that it would be harder to accept the reality of illness.¹⁴ For example, dysfunctional denial may delay help-seeking requests for serious illness symptoms, thus postponing treatments,¹⁵ or it could imply poor compliance with treatment or even involve an inadequate attitude of the caregiver in supporting the patient.^16,17 Furthermore, denial needs to be distinguished from conscious, aware, and deliberate avoidance of the threatening condition.¹⁸ It is important for health professionals to assess illness denial with reliable tools, but there is difficulty in retrieving specific and useful tools for this purpose – as underlined in previous studies.¹⁸ Indeed, the literature reports that semi-structured interviews¹⁹ or self-report questionnaires²⁰ have addressed only specific disorders and often have assessed something ambiguous instead of proper denial. The Illness Denial Questionnaire (IDQ)¹⁸ was created with the aim to assess denial in relation to illness, in patients and in caregivers and independently of the specific kind of illness or disturbance involved. The IDQ is a self-report questionnaire consisting of 24 dichotomous items (true/false) that show good psychometric properties and a three-factor structure. The IDQ assesses three dimensions: “Denial of negative emotions,” “Resistance to change,” and “Conscious avoidance.” “Denial of negative emotions” refers to the emotional reactivity, a subjective dimension, related to the individual’s emotional life and to its regulation. This dimension encompasses different emotions and feelings, such as sadness, insecurity, worry, and anger.²¹ “Resistance to change” concerns the behavioral efforts necessary to manage illness, and it is aimed at assessing the practical changes and modifications in one’s lifestyle in order to adapt to a new situation (eg, medications, therapies). Such transformation and reorganization mainly concern individual behaviors and habits.²² “Denial of negative emotions” and “Resistance to change” have been found to be the core components of denial.¹⁸ Besides these, even a third dimension is present, namely “Conscious avoidance.” This corresponds to a later and less severe critical stage of the illness elaboration process in which awareness is reached. At this point, acceptance of the illness’s existence can begin.²³ Despite the validity of the conscious avoidance construct, it could be confounding to measure it together with denial since they are well-distinguished concepts: denial implies a lack of awareness and is a preliminary phase characterized by the removal of unpleasant material from consciousness; whereas in conscious avoidance, awareness is present but the individual voluntarily avoids facing a threatening situation.^23,24 Furthermore, “Conscious avoidance” has proven to be independent and different from the core aspects of illness denial – both theoretically and psychometrically.¹⁸ Illness denial represents a more severe clinical issue than conscious avoidance, and it is also more difficult to assess given the lack of awareness. Thus, denial represents a critical issue for clinical practice, and the present work is specifically aimed to assess it. Given this premise, the present study is focused on the central aspects of illness denial: “Denial of negative emotions” and “Resistance to change.” Despite their interconnection, they concern two well-distinguished areas of the individual’s life: emotional and behavioral. Medical settings require short and reliable tools given their undoubted advantages: easy administration, suitable integration in longer routine assessment batteries, suitable administration to subjects with reduced attention ability, and rapid administration in medical wards such as waiting rooms.^25,26 Brief tools can be used for monitoring the individual’s progress over time, thus informing clinical decisions. The IDQ requires a time-consuming administration and a long scoring procedure, thus making it difficult for health professionals to immediately understand the actual denial condition of the patient/caregiver. To our knowledge, nowadays, brief tools specifically measuring illness denial are lacking. For these reasons, a short form of the IDQ is needed in order to conduct an efficient and valid assessment of illness denial. Furthermore, since denial affects patients and caregivers, a unique assessment tool suitable for both is desirable. This paper is aimed at developing and validating a brief version of the IDQ, which will be called Illness Denial Questionnaire–Short Form (IDQ-SF). In Study 1, the factor structure of the IDQ considering the two dimensions of “Denial of negative emotions” and “Resistance to change” was tested via confirmatory factor analysis (CFA). Then, the properties of the items were evaluated with the Rasch model analysis²⁷ in order to select the items with the best psychometric properties for the IDQ-SF. In Study 2, the factor structure and psychometric properties of the IDQ-SF were investigated. Study 1 Materials and methods Participants In line with the previous study,¹⁸ participants were enrolled in two rehabilitation hospitals of IRCCS Maugeri Scientific Institutes in Veruno and Pavia, Italy. The sample comprised 201 participants (56% females) with an overall mean age of 56.14 years (SD=12.80). Among the participants, 118 were inpatients (45% females; for females: mean age=56.08, SD=12.42; for males: mean age=58.72, SD=10.20). The remaining 83 subjects were caregivers (71.08% females; for females: mean age=53.17, SD=14.62; for males: mean age=56.63 years, SD=14.37). Caregivers displayed different kinds of relationship with patients: 47 spouses/living-with (56.6%), 1 partner/not-living-with (1.2%), 6 parents (7.2%), 21 sons/daughters (25.3%), and 8 in other types of relationships (9.6%). The participants, patients and caregivers, were retrieved from several medical wards: cardiology (26.37%), neurology (24.88%), oncology (22.40%), nephrology (12.94%), and pneumology (7.5%). More information about patients and caregivers is reported in Table 1. Table 1 Characteristics of the samples used in studies 1 and 2 [/RAW_REF_TEXT] Instruments The IDQ¹⁸ was administered to participants who were asked to express in a dichotomous form (false=0; true=1) if their experience was represented by each of the 24 statements. The IDQ evaluates three dimensions: “Denial of negative of emotions” (7 items; eg, “This disorder/disease frightens me”), “Resistance to change” (9 items; eg, “Nothing in my life will change on account of this disorder/disease”), and “Conscious avoidance” (8 items; eg, “I try not to pay any attention to my disorder/disease”). The first two dimensions were found to express the real form of denial, while the latter dimension of “Conscious avoidance” represented a more advanced stage of the illness elaboration process.¹⁸ After recoding the reverse items, higher scores on the IDQ expressed higher levels of denial. A psychologist introduced the study. All participants gave informed consent to use their data in anonymous form. The ethical committee of the Maugeri Scientific Institute discussed and approved the study. Statistical analyses In the first part of Study 1, the IDQs factor structure was tested through CFA, and reliability and validity were evaluated. Only the two main factors of denial were considered: “Denial of negative emotions” and “Resistance to change.”¹⁸ The diagonally weighted least squares (DWLS) estimator was used²⁸ given its suitability for categorical data. The goodness-of-fit was evaluated considering chi square, the χ²/df ratio, the comparative fit index (CFI),²⁹ the Tucker–Lewis index (TLI),³⁰ the root mean square of approximation (RMSEA),³¹ and the adjusted goodness-of-fit statistic (AGFI).³² A non-significant chi square expresses adequate model fit. However, chi-square is sensible to sample-size.³³ The χ²/df ratio is a handy measure of fit,^34–36 with values smaller than 3 indicating good fit.³⁷ AGFI estimates the part of variance accounted for by the estimated population covariance;³⁸ values greater than 0.90 denote well-fitting models. The CFI and the TLI express the amount of variance and covariance accounted for by the model compared with an unstructured baseline model. These indexes are unaffected by sample size.^39,40 Desirable CFI and TLI values are above 0.95, although values between 0.90 and 0.95 are acceptable.²⁹ The RMSEA describes fit per degrees of freedom of the model; values between 0.05 and 0.08 denote acceptable fit³⁹ and values lower than 0.05 express good fit.⁴¹ Reliability was evaluated for each of the two dimensions using Cronbach’s α.⁴² Values≥0.7 suggest acceptable-to-high scale reliability. Furthermore, in order to create the reduced form of the questionnaire, a Rasch model analysis was used to select the items with the best psychometric properties.^27,43,44 There is a long history of applications of Rasch models in the medical field.^45–48 Given the specific measurement properties of Rasch models, they have been widely used for the development of brief measurement.^49–53 Rasch models characterize the responses of persons to items as a function of person and item measures. These measures pertain to the level of a quantitative latent trait possessed by a person or item, and their specific meaning relies on the subject of the assessment. In educational assessments, for instance, person measures indicate the ability of persons, and item measures indicate the difficulty of items. In health status assessments, person measures indicate the health of persons, and item measures indicate the severity of items. In the context of IDQs, providing a “true” answer to an item is analogous to stating the presence of the measured latent trait (ie, denial). The Rasch model is usually applied to evaluate psychometric tests since it is centered on the study of items and their properties. Items are ordered according to their severity (location or beta): items with less “true” responses are more severe, and items with more “true” responses are less severe. Severity is represented on a scale with mean zero – the higher the values, the higher the severity. Items should cover all the distinct levels of the trait of interest in order to properly measure it. Items covering the same level of the latent construct are redundant and can thus be dropped. An adequate coverage of the severity continuum suggests content representativeness and construct validity.⁵⁴ Fit indices show how well the data fit the model, and their technical quality is evaluated. Item fit addresses the extent to which an item is related to the rest of the scale. Unfitting items may be considered for elimination. A valid measurement tool should cover different levels of the trait continuum and should not contain items with differential item functioning⁵⁵ (DIF). In the presence of DIF, comparisons among groups are questionable: they should only be qualitative and not quantitative.⁴⁵ The absence of DIF items allows quantitative comparisons among scores of different groups to be made.⁵⁴ Item reduction strategy Given that unidimensionality is an assumption of the Rasch model, the two dimensions of denial were separately analyzed by fitting two Rasch models, one for “Denial of negative emotions” and the other for “Resistance to change.” Given the equal distribution of items across the components, the same number of items was selected for each dimension. The length of the short form was not a priori defined but was suggested by statistical analyses. A minimum of 4 items per dimension was set in order not to weaken the scale too much. A two-step procedure was applied in order to create the short form: a) poorly functioning items were dropped; b) then, only the best items were selected for the short-form. First of all, DIF was analyzed to test the invariance of each item across different groups of subjects (patients and caregivers) and across sex (females and males). An item exhibiting DIF is differently answered by subjects with the same ability level but from different groups, and such nuisance factors affect goodness of measurement.⁵⁶ DIF results were considered in the selection of items since it is preferable to drop items showing DIF with large effect size⁵⁵ in order to obtain a test suitable for different subjects across sex and across groups (ie, patients and caregivers). Uniform and nonuniform DIF were considered: DIF is uniform if the item–group interaction is independent of the subjects’ ability level, while DIF is nonuniform when the item functioning is related to the subjects’ ability level.⁵⁷ Then, Rasch fit indices were used to evaluate item properties. Good fit indices mean that a certain item fits to the Rasch model’s expectations based on item difficulties and subjects’ ability level. Infit and outfit are sensitive to unexpected responses given a certain person’s trait level. Infit is more precise and detects unexpected responses to items that are close to a person’s trait level, while outfit perceives “bigger” unexpected responses to items that are far from the subject’s trait level. Fit indices rely on the mean square of the standardized residuals for items, which are not sample-size dependent. The expected value for fit indices is 1, but recommended values should not exceed 1.4 and might not be lower than 0.7.⁵⁸ Values above 1.4 suggest that there is unexplained variance (ie, underfit), while values lower than 0.7 mean that there is redundancy among the items (ie, overfit). Items showing outfit values over 1.4 were dropped, and items with fit indices closer to 1 were selected to be included in the short form. Furthermore, the item severity (beta) was considered in order to select the items that allowed for covering the entire continuum, thus providing a well-graded tool assessing different levels of the measured dimension without redundant items. Finally, the clinical meaning of each item was critically discussed by expert clinicians and was then considered to wisely drive the item choice in order to assess all the relevant clinical areas. All the data analyses were performed with R statistical environment (v. 3.5.0).⁵⁹ CFA was performed by using the “lavaan“ package (v. 0.6.3),⁶⁰ while Rasch analyses were conducted with TAM (v. 2.12.18)⁶¹ and “difR“ (v. 5.0)⁵⁷ packages. Results Results from the CFA on the two dimensions of denial showed a two-factor structure with good fit indices. Despite the chi-square fit reaching statistically significant values [χ²=208.747; df=103; p62 suggested the acceptability of the model (χ²/df=2.026;35 AGFI further suggested a well-fitting model (AGFI=0.911). As shown in Table 2, all the items significantly loaded on the intended factor (p Table 2 Factor loadings of the confirmatory factor analysis in Study 1 [/RAW_REF_TEXT] Two Rasch models were separately fitted for the two dimensions of denial. Table 3 reports the parameters considered in the selection process. After fitting the Rasch model on the “Denial of negative emotions” items, in the first step of selection all critical items showing DIF were dropped: Item#2 and Item#17 showed DIF between patients and caregivers; Item#10 showed DIF across sex. In the second step, the remaining 4 items showed acceptable infit and outfit values: none exceeded 1.4 or was under the lower threshold. Those items well covered the latent trait continuum, and their clinical content was considered relevant and meaningful. Thus, the items for the short form were: Item#6, Item#7, Item#21, and Item#23. Given these results, the length for each scale was settled at 4 items. Table 3 Selection parameters from Rasch analysis [/RAW_REF_TEXT] A second Rasch model was fitted on the “Resistance to change” items. In the first step of selection, no item showed DIF across sex, and only one item (Item#4) showed critical DIF across patients and caregivers and thus was dropped. In the second step, all the 8 remaining items showed satisfactory Rasch fit indices. Among them, the items with fit indices closer to the expected value of 1 were chosen. Also, the difficulty (beta parameter) and the clinical connotation were considered in the selection. Item#12 was chosen given its specific clinical meaning. The items in the couples Item#5–Item#13 and Item#11–Item#16 showed similar severity, but given the clinical meaning and the fit values, the items Item#5 and Item#16 were preferred. Finally, items Item#15 and Item#22 showed the same severity, but Item#22 was preferred given its clinical meaning. Thus, the other 4 items selected for the short form were Item#5, Item#12, Item#16, and Item#22. The resulting IDQ-SF is a brief tool aimed at assessing illness denial. It is composed of 8 items, 4 from “Denial of negative emotions” (Item#6, Item#7, Item#21, Item#23) and 4 from “Resistance to change” (Item#5, Item#12, Item#16, Item#22). Study 2 Study 2 was aimed at testing in a new sample the validity and the psychometric properties of the IDQ-SF developed in Study 1. Materials and methods Participants This sample comprised 202 participants (58% females; mean age=55.93, SD=13.88). The inpatients group counted 113 subjects (47% females; for females: mean age=58.60, SD=13.00; for males: mean age=59, SD=12.93). The caregiver group was composed of 89 subjects (71.91% females; for females: mean age=52.02, SD=13.36; for males: mean age=52.88, SD=16.68). Caregivers showed different relationships with patients: 49 spouses/living-with (55.1%); 1 partner/not-living-with (1.1%); 4 parents (4.5%); 23 sons/daughters (25.8%); and 12 in another kind of relationship (13.5%). Patients and caregivers were enrolled from different medical wards: oncology (26.73%), neurology (24.26%), cardiology (23.27%), nephrology (16.34%), and pneumology (3.96%). Descriptive statistics are reported in Table 1. After a short presentation made by a psychologist, all participants signed the informed consent allowing their data to be used in anonymous form. The study was discussed and approved by the ethical committee of the Maugeri Scientific Institute. Instruments Participants were presented with the IDQ-SF – developed in Study 1. They were asked to answer the 8 dichotomous items of the questionnaire with true or false according to their experience. The Anxiety and Depression Questionnaire–Reduced Form (AD-R) was used for evaluating state anxiety and depression symptoms.⁶³ It is a reduction of the State-Trait Anxiety Inventory⁶⁴ and the Depression Questionnaire.⁶⁵ The anxiety subscale measures state anxiety with 10 items on a 4-level-rating scale (not at all=1, somewhat=2, moderately so=3, and very much so=4). The total score ranges between 0 and 40. The depression subscale measures depression symptoms with 15 dichotomous items (no=0; yes=1). The total score of this scale ranges from 0 to 15. Statistical analyses The factorial structure of IDQ-SF was studied via CFA. Given the results from Study 1 and from previous literature,¹⁸ a two-factor structure was hypothesized. The DWLS was used as an estimator, and several indicators were considered to evaluate model fit: χ², χ²/df, CFI, TLI, RMSEA, and AGFI. Internal consistency was evaluated using Cronbach’s α. Concurrent validity was studied by correlating the IDQ-SF with the anxiety and depression subscales of the AD-R questionnaire. Results The CFA on the IDQ-SF tested the two-factor model where 4 items saturated in the factor “Denial of negative emotions” and 4 items saturated in the factor “Resistance to change.” Fit indices showed that the two-factor structure of the IDQ-SF provided a good depiction of the item responses: χ² (19)=28.408; p=0.076. The χ²/df ratio of the model suggested the acceptability of the model (χ²/df=1.49;Table 4. Internal consistency was good for “Denial of negative emotions” (α=0.71) and for “Resistance to change” (α=0.58). Table 4 Confirmatory factor analysis factor loadings of the Illness Denial Questionnaire-Short Form [/RAW_REF_TEXT] Concerning concurrent validity, correlations were calculated separately for patients and caregivers. The IDQ-SF total score showed a negative moderate correlation with anxiety subscales (patients=–0.57; caregivers=–0.45) and depression subscales (patients=–0.48; caregivers=–0.39). More information is provided in Table 5. Table 5 Correlations between the Anxiety and Depression Questionnaire-Reduced Form and the Illness Denial Questionnaire-Short Form [/RAW_REF_TEXT] General discussion The present paper aimed at developing and validating a short form of the IDQ for patients and caregivers, considering the core dimensions of “Denial of negative emotions” and “Resistance to change.” Study 1 was aimed at developing the short form. Results of this study confirmed the validity and reliability of the IDQ, as composed of these two related but different dimensions. According to hypotheses and in line with the previous study, results of CFA confirmed that these dimensions represent two factors concerning different aspects of the individual’s life: one on the behavioral level and the other on the emotional level. Results of CFA were an introductory phase for the item reduction in which a Rasch model was independently fitted on the items of each of the two denial dimensions. Rasch models represent valuable tools for the development and validation of brief assessment instruments. After considering the results of several statistics, Rasch analysis led to the selection of 4 items of “Denial of negative emotions” and 4 items of “Resistance to change,” thus leading to the IDQ-SF, a reduced version consisting of 8 items that were invariant across patients and caregivers. Thanks to the Rasch model properties, the items of the IDQ-SF are presented on a growing level of denial, thus allowing clinicians to immediately visualize if denial is present and its severity. If Rasch had not been used, poorly functioning items could have been included in the short form. Study 2 aimed at testing the validity of the short-form in a new sample: the factor structure of IDQ-SF was confirmed by a second CFA. The IDQ-SF also showed good psychometric properties, and, as found in the original form, it displayed negative correlations with anxiety and depression symptoms. This result makes sense from a theoretical point of view: emotions, such as anxiety and depression, are suppressed by the denial defense mechanism.^18,66 The IDQ-SF may have useful applications in the clinical field. By considering item severity, the answers of a subject can be evaluated to provide meaningful information to clinicians and health professionals. The number and the severity of endorsed items could indicate the psychological risk of a person and the urgency of clinical intervention. This study is not free of limitations, and its results should be considered object of discussion by future studies. Our sample mainly involved patients affected by chronic diseases in the rehabilitation setting and their caregivers. Future research should consider acute phases of illness and post-rehabilitation conditions.^67–70 An interesting future research topic would be to investigate the relationship among illness denial and other psychological conditions, such as well-being, stress, and caregivers’ strain. Further studies are needed to extend or disconfirm our results and to deepen further aspects of the denial and illness elaboration process in order to improve the efficacy and quality of the offered clinical treatment. Conclusion Illness denial represents a critical issue in clinical practice, not always evenly defined and thus difficult to assess. In a previous study, denial was defined as a process in which one’s perspective may fluctuate during the course of the illness before its acceptation. The IDQ-SF represents a brief tool focusing on denial’s core dimensions referring to the emotional experience and behavioral lifestyle change as previously emerged. Given its handy structure, the IDQ-SF’s administration is fast, fitting for patients with low attention span, and also suitable for integration in longer assessment batteries or to monitor the individual’s progress over time. Its interpretation and scoring are easier for clinicians, allowing an immediate depiction of the patient’s/caregiver’s intensity of denial, thus favoring the appropriate clinical intervention. In fact, psychological interventions need to promote the illness elaboration process, keeping in account that it fluctuates from denial to awareness to acceptance. The negative correlation among denial, anxiety, and depression⁶⁶ confirms that the individuals are not ready to cope with the illness in an effort to protect themselves from the perceived threat: this suggests a cautious approach by the clinicians in order to avoid even worse reactions and to help along the elaboration process of the illness. Even if more studies are needed to deepen the role of denial in the illness elaboration process, the IDQ-SF may be a starting point for its assessment and in catalyzing the clinical and scientific discussion about this intriguing topic. Abbreviations IDQ, Illness Denial Questionnaire; IDQ-SF, Illness Denial Questionnaire-Short Form; AD-R, Anxiety and Depression Questionnaire-Reduced Form; DIF, Differential Item Functioning. Ethics approval and informed consent The present manuscript includes two studies on human subjects. Informed consent was retrieved by all participants, and both studies were approved by the Ethical Committee of Maugeri Scientific Institutes (Pavia); protocol number: 974CE. Data availability The data generated and analyzed in the current study are property of Maugeri Scientific Institutes S.p.A. Restrictions apply to the availability of these data, which were used under license for the current study and so are not publicly available. Disclosure The authors report no conflicts of interest in this work. References [RAW_REF_TEXT] 1. Eisner MD, Blanc PD, Yelin EH, et al. Influence of anxiety on health outcomes in COPD. Thorax. 2010;65(3):229–234. doi:10.1136/thx.2009.126201 [/RAW_REF_TEXT] [RAW_REF_TEXT] 2. Levitt EE. A note on “MMPI scale development methodology. J Pers Assess. 1978;42(5):503–504. doi:10.1207/s15327752jpa4205_10 [/RAW_REF_TEXT] [RAW_REF_TEXT] 3. McGrath RE, O’Malley WB. The assessment of denial and physical complaints: the validity of the Hy scale and associated MMPI signs. J Clin Psychol. 1986;42(5):754–760. doi:10.1002/1097-4679(198609)42:53.0.CO;2-O [/RAW_REF_TEXT] [RAW_REF_TEXT] 4. Strachey J. The Ego and the Id: The Standard Edition of the Complete Psychological Works of Sigmund Freud. New York, NY: Norton & Company; 1960. doi:10.1177/105382590602900110 [/RAW_REF_TEXT] [RAW_REF_TEXT] 5. O’Neil MK, Akhtar S. On Freud’s Negation. Karnac Books; 2011. Available from: https://books.google.it/books?hl=it‎&id=AA-37hl5X7EC&oi=fnd&pg=PR7&dq=o+neill+2011+on+freud+s+negation&ots=fIYXdMCTbj&sig=Wg03plaxR6-FOlUw29q8qPWy1G4&redir_esc=y#v=onepage&q&f=false.AccessedJanuary16, 2019 . [/RAW_REF_TEXT] [RAW_REF_TEXT] 6. Watson M, Greer S, Blake S, Shrapnell K. Reaction to a diagnosis of breast cancer relationship between denial, delay and rates of psychological morbidity. Cancer. 1984;53(9):2008–2012. doi:10.1002/1097-0142(19840501)53:93.0.CO;2-B [/RAW_REF_TEXT] [RAW_REF_TEXT] 7. Aitken-Swan J, Easson E. Reactions of cancer patients on being told their diagnosis. Br Med J. 1959; 1:779–783. doi: 10.1136/bmj.1.5124.779 [/RAW_REF_TEXT] [RAW_REF_TEXT] 8. Livneh H. Psychosocial adaptation to chronic illness and disability: a conceptual framework. Rehabil Couns Bull. 2001;44(3):151–160. doi:10.1177/003435520104400305 [/RAW_REF_TEXT] [RAW_REF_TEXT] 9. Livneh H, Antonak RF. Psychosocial adaptation to chronic illness and disability: a primer for counselors. J Couns Dev. 2005;83(1):12–20. doi:10.1002/j.1556-6678.2005.tb00575.x [/RAW_REF_TEXT] [RAW_REF_TEXT] 10. Telford K, Kralik D, Koch T. Acceptance and denial: implications for people adapting to chronic illness: literature review. J Adv Nurs. 2006;55(4):457–464. doi:10.1111/j.1365-2648.2006.03942.x [/RAW_REF_TEXT] [RAW_REF_TEXT] 11. Chiao CY, Wu HS, Hsiao CY. Caregiver burden for informal caregivers of patients with dementia: a systematic review. Int Nurs Rev. 2015;62(3):340–350. doi:10.1111/inr.12194 [/RAW_REF_TEXT] [RAW_REF_TEXT] 12. Afrell M, Biguet G, Rudebeck CE. Living with a body in pain - between acceptance and denial. Scand J Caring Sci. 2007;21(3):291–296. doi:10.1111/j.1471-6712.2007.00475.x [/RAW_REF_TEXT] [RAW_REF_TEXT] 13. Stroebe M, Boelen PA, Van Den HM, Stroebe W, Salemink E. Ruminative coping as avoidance. Eur Arch Psychiatry Clin Neurosci. 2007;257(8):462–472. doi:10.1007/s00406-007-0746-y [/RAW_REF_TEXT] [RAW_REF_TEXT] 14. Lazarus RS. Coping theory and research: past, present, and future. Psychosom Med. 1993;55:234–247. doi:10.1097/00006842-199305000-00002 [/RAW_REF_TEXT] [RAW_REF_TEXT] 15. Covino JM, Stern TW, Stern TA. Denial of cardiac illness: consequences and management. Prim Care Companion CNS Disord. 2011;13(5). doi:10.4088/PCC.11f01166 [/RAW_REF_TEXT] [RAW_REF_TEXT] 16. Rammohan A, Rao K. Burden and coping in caregivers of persons with schizophrenia. Indian J Psychiatry. 2002;44(3):220–227. [/RAW_REF_TEXT] [RAW_REF_TEXT] 17. Ratti MM, Rossi A, Delli Zotti GB, Sarno L, Spotti D. Social support, psychological distress and depression in hemodialysis patients. Psicol Della Salut. 2017;(1):112–122. doi:10.3280/PDS2017-001006 [/RAW_REF_TEXT] [RAW_REF_TEXT] 18. Ferrario SR, Giorgi I, Baiardi P, et al. Illness denial questionnaire for patients and caregivers. Neuropsychiatr Dis Treat. 2017;13:909–916. doi:10.2147/NDT.S128622 [/RAW_REF_TEXT] [RAW_REF_TEXT] 19. Vos MS, Putter H, Leurs A, Rooijmans HGM, de Haes HCJM, van Houwelingen HC. The denial of cancer interview: development and first assessment of psychometric properties in lung cancer patients. Patient Educ Couns. 2007;67(1–2):224–234. doi:10.1016/j.pec.2007.03.019 [/RAW_REF_TEXT] [RAW_REF_TEXT] 20. Vanderdeycken W, Vanderlinden J. Denial of illness and the use of self‐reporting measures in anorexia nervosa patients. Int J Eat Disord. 1983;2(4):101–107. doi:10.1002/1098-108X(198322)2:43.0.CO;2-U [/RAW_REF_TEXT] [RAW_REF_TEXT] 21. Bowman GS. Emotions and illness. J Adv Nurs. 2001;34(2):256–263. doi:10.1046/j.1365-2648.2001.01752.x [/RAW_REF_TEXT] [RAW_REF_TEXT] 22. Gignac MAM, Cott C, Badley EM. Adaptation to chronic illness and disability and its relationship to perceptions of independence and dependence. J Gerontol - Psychol Sci Soc Sci. 2000;55(6):362–372. doi:10.1093/geronb/55.6.P362 [/RAW_REF_TEXT] [RAW_REF_TEXT] 23. Lazarus RS. Fifty Years of the Research and Theory of R. S. Lazarus: An Analysis of Historical and Perennial Issues. London: Routledge Sources in History, Psychology Press; 1998. doi:10.1097/00001888-200009000-00007 [/RAW_REF_TEXT] [RAW_REF_TEXT] 24. Roth S, Cohen LJ. Approach, avoidance, and coping with stress. Am Psychol. 1986;41(7):813–819. [/RAW_REF_TEXT] [RAW_REF_TEXT] 25. Paulhus DL, Vazire S. The self-report method. In: Robins RW, Krueger RC, editors. Handbook of Research Methods in Personality Psychology. New York: Guilford; 2005:224–239. doi:10.1016/B0-08-045044-X/00117-6 [/RAW_REF_TEXT] [RAW_REF_TEXT] 26. Vidotto G, Ferrario SR, Bond TG, Zotti AM. Family strain questionnaire - short form for nurses and general practitioners. J Clin Nurs. 2010;19:275–283. doi:10.1111/j.1365-2702.2009.02965.x [/RAW_REF_TEXT] [RAW_REF_TEXT] 27. Rasch G. Probabilistic Models for Some Intelligence and Achievement Tests. Chicago: University of Chicago; 1980. doi:10.1016/S0019-9958(61)80061-2 [/RAW_REF_TEXT] [RAW_REF_TEXT] 28. Jöreskog KG, Sörbom DL. LISREL 7: User’s Reference Guide. Chicago: Scientific Software International; 1989. [/RAW_REF_TEXT] [RAW_REF_TEXT] 29. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107:238–246. doi:10.1037/0033-2909.107.2.238 [/RAW_REF_TEXT] [RAW_REF_TEXT] 30. Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1–10. doi:10.1007/BF02291170 [/RAW_REF_TEXT] [RAW_REF_TEXT] 31. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors.Testing Structural Equation Models. Newbury Park, CA: Sage Focus Editions; 1993:156. [/RAW_REF_TEXT] [RAW_REF_TEXT] 32. Jöreskog K, Sörbom D. Lisrel VI Users’ Guide. Mooresville (IL): Scientific Software; 1984. [/RAW_REF_TEXT] [RAW_REF_TEXT] 33. Bentler PM, Bonett DG. Significance tests and goodness of fit in the analysis of covariance structures. Psychol Bull. 1980;88(3):588–606. doi:10.1037/0033-2909.88.3.588 [/RAW_REF_TEXT] [RAW_REF_TEXT] 34. Marsh HW, Hocevar D. Application of confirmatory factor analysis to the study of self-concept. first- and higher order factor models and their invariance across groups. Psychol Bull. 1985;97(3):562–582. doi:10.1037/0033-2909.97.3.562 [/RAW_REF_TEXT] [RAW_REF_TEXT] 35. Hoyle RH. Handbook of Structural Equation Modeling. New York: Guilford Press; 2012. doi:10.4135/9781412961288.n69 [/RAW_REF_TEXT] [RAW_REF_TEXT] 36. Pietrabissa G, Rossi A, Simpson S, et al. Evaluation of the reliability and validity of the Italian version of the schema mode inventory for eating disorders: short form for adult with dysfunctional eating behavior. Eat Weight Disord - Stud Anorexia, Bulim Obes. 2019:1–13. doi:10.1007/s40519-019-00644-5 [/RAW_REF_TEXT] [RAW_REF_TEXT] 37. Wheaton B, Muthén B, Alwin DF, Summers GF. Assessing reliability and stability in panel models. Sociol Methodol. 1977;8:84–136. doi:10.2307/270754 [/RAW_REF_TEXT] [RAW_REF_TEXT] 38. Tabachnick BG, Fidell LS. Using Multivariate Statistics.7th ed. Boston: Pearson; 2007. doi:10.1037/022267 [/RAW_REF_TEXT] [RAW_REF_TEXT] 39. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model a Multidiscip J. 1999;6(1):1–55. doi:10.1080/10705519909540118 [/RAW_REF_TEXT] [RAW_REF_TEXT] 40. Hu L, Bentler PM. Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychol Methods. 1998;3(4):424–453. doi:10.1037/1082-989X.3.4.424 [/RAW_REF_TEXT] [RAW_REF_TEXT] 41. Browne MW, Cudeck R. Single sample cross-validation indices for covariance structures. Multivariate Behav Res. 1989;24(4):445–455. doi:10.1207/s15327906mbr2404_4 [/RAW_REF_TEXT] [RAW_REF_TEXT] 42. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334. doi:10.1007/BF02310555 [/RAW_REF_TEXT] [RAW_REF_TEXT] 43. Rasch G. Probabilistic models for some intelligence and achievement tests. Copenhagen Danish Inst Educ Res. 1960. doi:10.1016/S0019-9958(61)80061-2 [/RAW_REF_TEXT] [RAW_REF_TEXT] 44. Noventa S, Stefanutti L, Vidotto G. An analysis of item response theory and rasch models based on the most probable distribution method. Psychometrika. 2014;79(3):377–402. doi:10.1007/s11336-013-9348-y [/RAW_REF_TEXT] [RAW_REF_TEXT] 45. Clauser BE, Ross LP, Nungester RJ, Clyman SG. An evaluation of the Rasch model for equating multiple forms of a performance assessment of physiciansʼ patient-management skills. Acad Med. 1997;72(Suppl. 1):76–78. doi:10.1097/00001888-199710001-00026 [/RAW_REF_TEXT] [RAW_REF_TEXT] 46. Fisher WP, Vial RH, Sanders CV. Removing rater effects from medical clerkship evaluations. Acad Med. 1997;72:443–444. doi:10.1097/00001888-199705000-00079 [/RAW_REF_TEXT] [RAW_REF_TEXT] 47. Da Dalt L, Anselmi P, Bressan S, et al. A short questionnaire to assess pediatric resident’s competencies: the validation process. Ital J Pediatr. 2013;39:41. doi:10.1186/1824-7288-39-41 [/RAW_REF_TEXT] [RAW_REF_TEXT] 48. Da Dalt L, Anselmi P, Furlan S, et al. Validating a set of tools designed to assess the perceived quality of training of pediatric residency programs. Ital J Pediatr. 2015;41:2. doi:10.1186/s13052-014-0106-2 [/RAW_REF_TEXT] [RAW_REF_TEXT] 49. Vidotto G, Carone M, Jones PW, Salini S, Bertolotti G. Maugeri respiratory failure questionnaire reduced form: A method for improving the questionnaire using the Rasch model. Disabil Rehabil. 2007;29(13):991–998. doi:10.1080/09638280600926678 [/RAW_REF_TEXT] [RAW_REF_TEXT] 50. Vidotto G, Bertolotti G, Carone M, et al. A new questionnaire specifically designed for patients affected by chronic obstructive pulmonary disease. The Italian Health Status Questionnaire. Respir Med. 2006;100(5):862–870. doi:10.1016/j.rmed.2005.08.024 [/RAW_REF_TEXT] [RAW_REF_TEXT] 51. Colledani D, Robusto E, Anselmi P. Development of a new abbreviated form of the Junior Eysenck Personality Questionnaire-Revised. Pers Individ Dif. 2018;120:159–165. doi:10.1016/j.paid.2017.08.037 [/RAW_REF_TEXT] [RAW_REF_TEXT] 52. Colledani D, Anselmi P, Robusto E. Using item response theory for the development of a new short form of the eysenck personality questionnaire-revised. Front Psychol. 2018;9:1834. doi:10.3389/fpsyg.2018.01834 [/RAW_REF_TEXT] [RAW_REF_TEXT] 53. Colledani D, Anselmi P, Robusto E. Using multidimensional item response theory to develop an abbreviated form of the Italian version of Eysenck’s IVE questionnaire. Pers Individ Dif. 2019;142:45–52. doi:10.1016/j.paid.2019.01.032 [/RAW_REF_TEXT] [RAW_REF_TEXT] 54. Smith EV. Evidence for the reliability of measures and validity of measure interpretation: a Rasch measurement perspective. J Appl Meas. 2001;2:281–311. [/RAW_REF_TEXT] [RAW_REF_TEXT] 55. Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Human Sciences.3rd ed. New York: Routledge; 2015. doi:10.1207/S15327574IJT013&4_10 [/RAW_REF_TEXT] [RAW_REF_TEXT] 56. Ackerman TA. A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. J Educ Meas. 1992;29:67–91. doi:10.1111/j.1745-3984.1992.tb00368.x [/RAW_REF_TEXT] [RAW_REF_TEXT] 57. Magis D, Béland S, Tuerlinckx F, de Boeck P. A general framework and an R package for the detection of dichotomous differential item functioning. Behav Res Methods. 2010;42(3):847–862. doi:10.1177/1359105315581516 [/RAW_REF_TEXT] [RAW_REF_TEXT] 58. Wright B, Linacre J, Gustafson J, Martin-Lof P. Reasonable mean-square fit values. Rasch Meas Trans. 1994;8:370. doi:10.1177/01461672012710004 [/RAW_REF_TEXT] [RAW_REF_TEXT] 59. R Core Team. R: a language and environment for statistical computing. Available from: http://www.R-project.org/. R Foundation for Statistical Computing, Vienna, Austria. doi:10.1007/978-3-540-74686-7 [/RAW_REF_TEXT] [RAW_REF_TEXT] 60. Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw. 2012;48(2):1–36. doi:10.18637/jss.v048.i02 [/RAW_REF_TEXT] [RAW_REF_TEXT] 61. Robitzsch A, Kiefer T, Wu M TAM: test analysis modules. R package version 2.12-18. 2018. doi:10.1177/0146621697211001 [/RAW_REF_TEXT] [RAW_REF_TEXT] 62. Jöreskog KG. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika. 1969;34(2):183–202. doi:10.1007/BF02289343 [/RAW_REF_TEXT] [RAW_REF_TEXT] 63. Moroni L, Bettinardi O, Vidotto G, et al. Scheda Ansia e Depressione forma ridotta: norme per l’utilizzo in ambito riabilitativo. Monaldi Arch Chest Dis. 2006;66(4):255–263. doi:10.4081/monaldi.2006.516 [/RAW_REF_TEXT] [RAW_REF_TEXT] 64. Spielberger CD, Gorsuch RL, Lushene RE. STAI manual for the state-trait anxiety inventory. Self-Evaluation Questionnaire. Manual. 1970;1–24. doi:10.1037/t06496-000 [/RAW_REF_TEXT] [RAW_REF_TEXT] 65. Bertolotti G, Michielin P, Vidotto G, Zotti AM, Sanavio E. Depression Questionnaire (DQ). In: Practitioner’s Guide to Empirical Based Measures of Depression. Norwell (MA): Kluwer Academic/Plenum Pubblishers; 2000:45–47. [/RAW_REF_TEXT] [RAW_REF_TEXT] 66. White KS, Pardue C, Ludbrook P, Sodhi S, Esmaeeli A, Cedars A. Cardiac Denial and psychological predictors of cardiac care adherence in adults with congenital heart disease. Behav Modif. 2016;40(1–2):29–50. doi:10.1177/0145445515613329 [/RAW_REF_TEXT] [RAW_REF_TEXT] 67. Kortte KB, Wegener ST. Denial of illness in medical rehabilitation populations: theory, research, and definition. Rehabil Psychol. 2004;49(3):187–199. doi:10.1037/0090-5550.49.3.187 [/RAW_REF_TEXT] [RAW_REF_TEXT] 68. Pietrabissa G, Manzoni GM, Rossi A, Castelnuovo G. The MOTIV-HEART study: a prospective, randomized, single-blind pilot study of brief strategic therapy and motivational interviewing among cardiac rehabilitation patients. Front Psychol. 2017;8:1–12. doi:10.3389/fpsyg.2017.00083 [/RAW_REF_TEXT] [RAW_REF_TEXT] 69. Jackson JB, Pietrabissa G, Rossi A, Manzoni GM, Castelnuovo G. Brief strategic therapy and cognitive behavioral therapy for women with binge eating disorder and comorbid obesity: a randomized clinical trial one-year follow-up. J Consult Clin Psychol. 2018;86(8):688–701. doi:10.1037/ccp0000313 [/RAW_REF_TEXT] [RAW_REF_TEXT] 70. Chan F, Da Silva Cardoso E, Chronister JA. Understanding Psychosocial Adjustment to Chronic Illness and Disability: A Handbook for Evidence-Based Practitioners in Rehabilitation. New York, NY: Springer Publishing Company; 2009. doi:10.1177/0034355210386240 [/RAW_REF_TEXT]