A protocol for the development and internal validation of a model to predict clinical response to antihistamines in urticaria patients

Abstract

Chronic urticaria causes a significant limitation to quality of life. In the literature, various studies can be found that have reviewed several clinical and laboratory markers, but none of these variables alone is sufficient to predict the patient's prognosis. In this study, we present a protocol to develop a prognostic model that can predict the clinical response of urticaria patients to antihistamines. This is a protocol for a bidirectional cohort study. Urticaria data will be routinely collected from a population of patients over 18 years old. A full multivariable logistic regression model will be fitted, following five steps: 1) Selection of predictive variables for the model; 2) Evaluation of the quality of the collected data and control of lost data; 3) Data statistical management; 4) Strategies to select the variables to include at the end of the model; 5) Evaluation of the performance of the different possible models (predictive accuracy) and selection of the best model. The performance and internal validation of the model will be assessed. Some clinical and paraclinical variables will be measured for further exploration.

About the Authors:

Jorge Sanchez

Contributed equally to this work with: Jorge Sanchez, Margarita Velasquez, Fabian Jaimes

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Current address: Allergy Center "IPS University" Clinic, Medellín, Colombia

Affiliation: Group of Clinical and Experimental Allergy, "IPS University" Clinic, University of Antioquia, Medellín, Colombia

ORCID logo http://orcid.org/0000-0001-6341-783X

Margarita Velasquez

Contributed equally to this work with: Jorge Sanchez, Margarita Velasquez, Fabian Jaimes

Roles Conceptualization, Formal analysis, Investigation, Writing – original draft, Writing – review & editing

Current address: Dermatology Center, "San Vicente" Hospital, Medellín, Colombia

Affiliation: Dermatological Research Center, University of Antioquia, Medellín, Colombia

ORCID logo http://orcid.org/0000-0001-8604-6488

Fabian Jaimes

Contributed equally to this work with: Jorge Sanchez, Margarita Velasquez, Fabian Jaimes

Roles Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Writing – review & editing

Affiliation: Internal Medicine Department, “San Vicente” Clinic, University of Antioquia, Medellín, Colombia

Introduction

Chronic spontaneous urticaria (CSU) is a disease characterized by the sudden appearance of hives and/or angioedema that can cause a significant limitation to quality of life, and it can be associated with different comorbidities such as hypothyroidism, depression, and anxiety. Despite the increased knowledge of its pathogenesis, the evolution of this disease in each patient is currently uncertain. Different studies have reviewed several clinical and laboratory markers to predict the evolution of the disease in terms of its duration or clinical treatment response [1–16].

Despite the fact that different variables have been identified, it seems that none of these variables alone is sufficient to adequately predict any of these outcomes. The construction of a model that includes the set of the most relevant variables could improve predictive accuracy.

The first lines of treatment in urticaria are based on the use of antihistamines; Several patients (40% to 60%) have clinical control with this therapy in conventional or higher doses [17, 18], but the rest of the patients need additional therapies such as the use of omalizumab or cyclosporine, which generates an increase in time and resource costs.

In this article, the following research question is posed: From the clinical characteristics and the IgE and IgG autoantibodies against self-proteins such as thyroperoxidase, could a predictive model be constructed to predict the clinical response to antihistamines in patients with urticaria? The creation of this prognostic model could help identify patients who will have a poor response to antihistamines, and define the early initiation of other therapies.

Therefore, the main objective of this study is to present a protocol to develop a prognostic model that allows the prediction of clinical response to treatment with antihistamines in adult patients with chronic spontaneous urticaria (Fig 1).

[Figure omitted. See PDF.]

Fig 1. Graphic summary illustrating the steps of the protocol.

https://doi.org/10.1371/journal.pone.0239962.g001

Seeking the best research performance in clinical fields and in favor of collaborative work in the scientific community, this protocol is openly available through this online publication.

Methods

This is a registered protocol, nevertheless we have ethics committee approved from University of Antioquia (Code F-017-00) and from "IPS Universitaria" clinic (Code 2019–2020).

Study design and source of data

This is an observational bidirectional cohort study. The first objective will be conducted in accordance with existing guidelines for model development and internal validation [19, 20] and reported in accordance with the “Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis” (TRIPOD) statement [21–24]. Two sources of recruitment will be used from two cities (Medellín and Bogotá) in Colombia:

1) Retrospective data: Patients from the URTICA cohort (URTICA cohort; Urticaria Research of Tropical Impact Control Assessment. Institutional approval IN40-2016, ClinicalTrials.gov identifier: NCT01940393). Those patients who meet the selection criteria for this project will be included if they have all the evaluation variables (predictive variables, outcome variable) and follow-up times.

We will only use retrospective data in case of not reaching a sufficient number of patients in the period of time for the prospective recruitment and in this case comparative analyzes of the retrospective and prospective groups will be made to evaluate the introduction of possible biases.

2) Prospective data: All new patients with a diagnosis of urticaria who attend one of the participating centers and meet the inclusion criteria will be invited to participate.

Patients will be recruited from records made from 7 December 2019 to 31 December 2021. Fig 2 shows the different follow-up periods and the procedures performed in each one.

[Figure omitted. See PDF.]

Fig 2. Collection of data.

Recruitment will be ambispective. Example 1: Patients from the URTICA [25] and CURE studies (Chronic Urticaria REgistry) who meet the selection criteria, for whom follow-up data are available, and for whom biological samples are available will be included in the study. Example 2: Patients who participate in the URTICA or CURE studies but have not yet been followed up for 2 years can also be included. Example 3: Patients who meet the selection criteria and are prospectively recruited from the participating institutions will be included.

https://doi.org/10.1371/journal.pone.0239962.g002

Participants and eligibility criteria

The reference population will be patients with chronic spontaneous urticaria, over 18 years of age, located in the cities of Medellín and Bogotá (Colombia), and attending dermatology and/or allergy services (secondary care level) at the following institutions: Medellín, Clinic “IPS Universitaria”, Hospital “San Vicente de Paul”, Clinic “Unidad Alergológica”; Bogotá, Clinic “Fundación Santa Fe, IPS UNIMEQ-ORL”. Because these are reference centers in health in the respective cities with hospital and outpatient medical care, the recruited patients will have different levels of severity. CSU medical protocol in these centers is based on the recommendations of international guidelines [27].

The patient inclusion age (≥18 years) is based on the epidemiological peak of the disease [6, 26]. The definition of spontaneous chronic urticaria will be according to the EAACI (European Allergy, Asthma and Clinical Immunology) criteria [27]: the presence of hives and/or angioedema for more than 6 weeks without an explanatory trigger (for example, medications or food). Patients should not have any medical contraindications for the beginning or maintenance of treatment with antihistamines (for example, allergy to the active substance or its excipients, intolerable adverse effects, etc.).

Those patients who suffer from a comorbidity that causes pruritus and could confuse diagnosis will be excluded (for example, dermatitis, chronic itching, scabies, peripheral neuropathy, etc.). Patients who have been treated with monoclonal anti-IgE or any other medication that may affect the evaluation of the clinical impact of antihistamines will not be included [27].

Outcome and predictors

The outcome variable will be nonresponse to antihistamines in conventional doses or, if necessary, after quadrupling the conventional daily dose for at least one month without obtaining a clinically effective response via the Urticaria Activity Score (UAS7 end <6 points, "control"; UAS7 end >7 points, "no control").

The selection of the model’s prognostic variables was made according to biological plausibility, an exhaustive review of possible variables, and the feasibility of their measurement and reproduction worldwide. We also considered some previous systematic reviews [28–31]. Based on these studies, the variables that we chose to include in the predictive model had different reasons: We include sex and age because they may be related to the outcome of interest but also because they help to control for other confounding factors that may be associated. The duration of the disease, the presence of angioedema, the severity at the onset of symptoms and the presence of comorbidities (inducible urticaria, autoimmunity, NSAIDs, hypersensitivity) seem to be associated with a longer duration of the disease and a lower response to antihistamine treatment.

Anxiety and depression can affect the control of the disease by its association with the severity of the symptoms and the way in which the patient perceives them. States that favor chronic inflammation also appear to be associated with a lower response to antihistamines, so we include variables associated with this state (CRP, body mass index, autoantigens). The predictive variables that we consider evaluating for the model are described in Table 1. In this study, considering the cohort design and that the outcome is easily objectifiable (formation of wheals or not), a blind assessment of the outcome to be predicted is not necessary. All predictors will be measured at the first appointment, prior to the start of continuous treatment with antihistamines.

[Figure omitted. See PDF.]

Table 1. Predictive variables to consider in the model.

https://doi.org/10.1371/journal.pone.0239962.t001

Sample size

Nonprobability convenience sampling will be carried out according to the order of arrival at clinical centers. The sample calculation will be according to the criterion of "events by variable" for predictive indices [32, 33]. This criterion is based on obtaining a number of outcomes for each predictive variable studied. Considering categorical variables that may require a greater number of patients to be evaluated and the frequency of the prognostic variables to be included in the model, we decided to include 20 subjects (potential outcomes) for each variable. Considering that 30 to 70% of patients achieve symptom control with antihistamines according to UAS7 (outcome of interest), and based on a previous study in the cities where the recruitment will take place [17], we consider that this outcome could occur in 50% of our population, so the formula applied for sample size would be the following:

n = 20*k/i where "k" is the number of predictive variables and "i" is the incidence of the outcome of interest: n = 20*15/0.5 = 600. Expecting a loss of 10% (60 patients), the minimal sample size would be 660 patients.

Missing data

The multiple imputation strategy will be applied to complete the possible missing data that we obtain from the variables as long as they do not exceed 10% of the data and the lost data can be assumed to be by random causality (when the probability that the data are lost is independent of the observed and unobserved characteristics of the sample); otherwise, if lost data of a variable exceed 10% or are not random, the variable should be removed from the model.

For this study we will use the interactive Monte Carlo and Markov chains (MCMC) method, since it provides ease of use when the loss pattern does not have characteristics of monotonic function.

Statistical analysis methods

For the analysis and construction of the model, once the information has been collected, missing or atypical data will be verified to avoid inconsistencies, and a descriptive analysis of the characteristics of the patients admitted to the institutions of interest will be carried out. The mean, standard deviation, and range will be used as summary measures when it comes to quantitative variables. For qualitative variables, frequency distributions and percentages will be used.

There are five steps to follow in the processing of the variables and the analysis of the data (Steps 1, 2, and 3 are the results of decisions made prior to the construction of the model) (Fig 3):

[Figure omitted. See PDF.]

Fig 3. Methodological summary.

Summary of the model according to the methodology used in the three main steps: 1) construction of the cohort, 2) development of the prognostic model, and 3) evaluation of model performance.

https://doi.org/10.1371/journal.pone.0239962.g003

Step 1: Selection of variables for the model.

The selection of the model’s prognostic variables was made according to biological plausibility, an exhaustive review of the variables that have been previously studied [12, 15, 28, 30, 31, 34–38], and the feasibility of their measurement in clinical practice (Table 1).

Step 2: Evaluation of the quality of collected data and management of lost data.

The steps to follow in case of requiring imputation will be the following:

1. Summary of missing values via graph and the percentage of missing data.

2. Patterns of missing values: The tabulation of missing data can indicate whether the losses follow a monotonic pattern and the type of imputation should be changed.

3. Generation of five different imputations.

4. Weighting of multiple imputations to obtain a single database.

Step 3: Data management.

Analysis of the collinearity assumption in continuous variables will be applied by means of the correlation matrix (where a value of r > 0.7 is considered to indicate collinear variables) or by the variance inflation factor (VIF > 5 indicating collinearity) and in categorical variables by means of the chi-square test of independence (relationship between categorical variables with p > 0.05 indicating collinearity).

The monotonic function relationship assumption will be evaluated in the continuous variables graphically using the Lowess function; those variables that by graphic criteria do not meet this assumption will be transformed or dichotomized.

Step 4: Strategies to select the variables to include at the end of the model.

For each of the variables that remain in the model, the degree of association with the dependent (outcome) variable (bivariate analysis) will be estimated by calculating the odds ratio (with its corresponding 95% confidence interval), and a multivariate regression model will be applied. To select the independent variables associated with the outcome of interest, the Wald statistic will be evaluated, considering p < 0.05 to include a variable as significant in the multivariate model and/or biological plausibility.

Step 5: Evaluation of model performance (predictive accuracy) and internal validation.

To compare the predictive capacity of the models, we will evaluate the discrimination and calibration: Discrimination will be evaluated in each of the models graphically and by using the c statistic and area under the ROC curve (Receiver Operating Characteristic). Calibration tells us about the goodness of fit of the model; we will use the Hosmer–Lemeshow hypothesis test (a value of p > 0.05 will indicate that the model has good calibration).

For the selection of the best model, several models will be built according to the inclusion and exclusion of variables. To compare these prognostic models, the data obtained from a bootstrap analysis will be used as a comparison sample (internal validation). Among the models, we will evaluate the discrimination improvement index, IDI (Integrated Discrimination Improvement), and the reclassification improvement index, NRI (Net Reclassification Improvement) [39].

The IDI linearly evaluates the change in the probability estimate of the outcome between the models. It will be calculated as the difference between the means of the estimated probabilities of the outcome of one model versus those estimated in the other, minus the same difference between those who did not have the outcome. In other words, the IDI represents the average improvement of a model in terms of the prediction of patients who actually have the outcome, removing what is worsened by the prediction of the result in patients who ultimately do not have it. The NRI considers changes between the different risk categories (success and nonsuccess). To estimate the categorical NRI, the outcome will be used and two comparison tables will be constructed separately according to whether or not the event of interest has been recorded in the patients.

The Akaike information criterion (AIC) and Bayesian information criterion (BIC) will also be evaluated (the lowest value obtained will define the best model) for the selection.

In case the final model has good performance, the specifications of the model, its limitations, and its interpretation will be explained in detail. The analyses described will be run using R, SPSS, and/or STATA software.

Model score and presentation format

The model score is closely tied to the presentation format. Ideally, we want to make a model with variables in their natural form (not transformed) that allows us to establish a risk score and offer it in the form of an electronic application available on computers and cell phones. In this way, the clinical prediction model provides an estimate of risk with a wide range (relatively low to relatively high risk), according to patient characteristics easy to use for the physician.

However, during the development of the model it may be necessary to carry out some transformations and according to the performance of the predictive model and the feasibility of its use for the clinician, we can consider dichotomizing the result variable and using other forms of presentation like example model, spreadsheet or score chart.

Limitations

The cohort design, realization by trained personnel, monthly interim analysis, and joint training of the different centers with the periodical monitoring that we plan to carry out will help to reduce selection, measurement, and confusion biases [40]. We can detect some limitations: the selection criteria and sampling of the retrospective cohort could result in differences with the prospective cohort; however, because the population, recruitment, and selection protocol are the same in the two studies and the time difference between the patients selected in the retrospective and prospective cohorts is less than one year, we consider that these risks are minimized.

Patients with CSU may have variable intensity of symptoms, and those with weaker symptoms are likely to have less motivation to participate. Although this is a problem that may affect the generalizability of the results and the applicability of the model in patients with mild symptoms, it has equal influence on the groups that result for the outcome of interest (responding or not to antihistamines), which allows to avoid imbalances in the interpretation.

The measurement of the laboratory variables will be carried out in duplicate; additionally, the measurements will be carried out in two laboratories, which will reduce the risk of error in the measurement of these variables.

Considering that the outcome of the study is to evaluate the clinical control achieved with antihistamines, it is possible that the group of subjects who achieve rapid control will be more tempted to leave the study than the group that does not obtain an adequate response. To control this bias, the importance of continuing in the study until the end of the same will be emphasized to the study participants at the time of recruitment; they will also be asked to agree to receive calls or visits, which will remind them of follow-up appointments and/or data collection.

Due to the fact that clinical control can be temporary, patients will be followed up over time (Fig 2) for a minimum of three months to define whether control is constant over time.

Ethical considerations

The investigation will be carried out in each center when authorization is obtained from the legal representative of each institution and the respective ethics committee. Each research subject will provide a written informed consent. In the event that a methodological change is required during the execution of the project, the amendments will be submitted again to the ethics committee.

In case of not obtaining the necessary number of patients in the recruitment period, we will use retrospective data from two previous projects that are in development and that have been approved by the institutional ethics committees. In these previous urticaria studies, each patient was asked for a written informed consent, where they authorized the use of the data for related projects with prior authorization from the ethics committee.

In this project, we will comply with the current national regulations on research and international regulations; we adhere to the recommendations of the Nuremberg code (1946), the UNESCO Universal Declaration on Bioethics and Human Rights (2005), and the Helsinki declaration (2013), and we will follow international ethical guidelines for health-related research (CIOMS 2018) in the aspects that correspond to this project.

Implications of this protocol

Although several variables that seem to be associated with the clinical response to antihistamines in CSU patients have been identified, at the moment, none is sufficient to make a correct prediction. Multivariable models could be a method to achieve through a set of variables what cannot be achieved through them separately. No multivariate models have been developed or validated to predict the response to antihistamines in CSU patients; our research has this objective and could benefit patients from different populations since this prediction model would allow for identifying those patients in whom antihistamine therapy is adequate and those patients who would require other therapies, reducing the use of unnecessary management. Conventional doses of antihistamines and increased dosages are the first two management steps according to current urticaria guidelines, and patients should wait up to four weeks to go from one treatment stage to another. In the patients who will fail to respond to the antihistamine, the predictive model would allow faster treatment adjustments, leading to better quality of life when controlling the disease, while on the other hand allowing better use of medical appointment resources, emergency consultations, and drug costs.

This research may also offer valuable insights into practical aspects of the clinical and molecular characteristics of urticaria since it allows for identifying possible endophenotypes of patients according to the clinical and laboratory characteristics associated with response (or lack thereof) to antihistamines.

The variables chosen as candidates for the predictive model come from several previous studies, some of which have been replicated in different populations and have biological plausibility, which makes them good candidates for the outcome of interest. The prospective recruitment of the study allows us to include the a priori candidate variables that we believe are most relevant as long as we meet the pre-established sample size defined by the ratio of number of events per variable, to avoid overestimation of the model.

According to the performance of the final model, it may be possible to evaluate reproducibility and transportability [19, 41]. This predictive model is intended to be applied in a population with CSU that is going to formally start its management with antihistamines. Since the model is initially being developed for use by dermatologists and allergists, it will later be necessary to evaluate its use by general practitioners.

Among the selected variables, some correspond to clinical and molecular variables; however, most of them are nonmodifiable. Therefore, in terms of prevention strategies, they have limited utility, but this does not affect the predictive capacity of the model and could facilitate the external reproduction process since they are not variables that are modified by environmental factors.

We hope that this study will also identify further research priorities for this novel and potentially valuable area of urticaria research; further hypothesis-free exploratory studies that investigate many factors may assist with the identification of new factors (biologically associated or not) that may help in management decisions and for monitoring purposes [42]. Furthermore, these types of studies may be used to update and improve the performance of the predictive model [43].

Finally, once the final results of the study will be published, seeking the greatest possible transparency, the source of the analysis database will be attached to the article but recognizing the possible restrictions that the participating centers and journal has in this regard (e.g., extension of supplementary material, type of authorized material, preserve the privacy of each patient).

In conclusion, we have developed a protocol for creating a multivariable prognostic model focused on predicting the response to antihistamines in patients with CSU. From this model, new lines of research can emerge and contribute both to the external validation of the model and to the construction of new models with different outcomes.

Acknowledgments

We thanks the technical support of Ridca Ardila and Juan Lpoez from University of Antioquia.

Citation: Sanchez J, Velasquez M, Jaimes F (2020) A protocol for the development and internal validation of a model to predict clinical response to antihistamines in urticaria patients. PLoS ONE 15(10): e0239962. https://doi.org/10.1371/journal.pone.0239962

References

1. Sánchez-Borges M, Caballero-Fonseca F, Capriles-Hulett A, González-Aveledo L, Maurer M. Factors linked to disease severity and time to remission in patients with chronic spontaneous urticaria. J Eur Acad Dermatol Venereol. 2017;31(6):964–71. pmid:28299827

2. Sussman G, Abuzakouk M, Bérard F, Canonica W, Oude Elberink H, Giménez-Arnau A, et al. Angioedema in chronic spontaneous urticaria is underdiagnosed and has a substantial impact: Analyses from ASSURE-CSU. Allergy. 2018;73(8):1724–34. pmid:29460968

3. Sánchez-Borges M, Caballero-Fonseca F, Capriles-Hulett A. Cofactors and comorbidities in patients with aspirin/NSAID hypersensitivity. Allergol Immunopathol (Madr). 2017;45(6):573–8. pmid:27865540

4. Sánchez Jorge J, Sánchez A, Cardona R. Prevalence of drugs as triggers of exacerbations in chronic urticaria. J Investig Allergol Clin Immunol. 2018:0.

5. Sánchez-Borges M, Acevedo N, Caraballo L, Capriles-Hulett A, Caballero-Fonseca F. Increased total and mite-specific immunoglobulin E in patients with aspirin-induced urticaria and angioedema. J Investig Allergol Clin Immunol. 2010;20(2):139–45. pmid:20461968

6. Sánchez J, Amaya E, Acevedo A, Celis A, Caraballo D, Cardona R. Prevalence of Inducible Urticaria in Patients with Chronic Spontaneous Urticaria: Associated Risk Factors. J Allergy Clin Immunol Pract. 2017;5(2):464–70. pmid:27838325

7. Ben-Shoshan M, Grattan CE. Management of Pediatric Urticaria with Review of the Literature on Chronic Spontaneous Urticaria in Children. J Allergy Clin Immunol Pract. 2018;6(4):1152–61. pmid:29550102

8. Weldon D. Quality of life in patients with urticaria and angioedema: assessing burden of disease. Allergy Asthma Proc. 2014;35(1):4–9. pmid:24433591

9. Maurer M, Weller K, Bindslev-Jensen C, Giménez-Arnau A, Bousquet PJ, Bousquet J, et al. Unmet clinical needs in chronic spontaneous urticaria. A GA2LEN task force report. Allergy. 2011;66(3):317–30. pmid:21083565

10. Asero R, Marzano AV, Ferrucci S, Cugno M. D-Dimer Plasma Levels Parallel the Clinical Response to Omalizumab in Patients with Severe Chronic Spontaneous Urticaria. Int Arch Allergy Immunol. 2017;172(1):40–4. pmid:28219067

11. Grzanka R, Damasiewicz-Bodzek A, Kasperska-Zajac A. Interplay between acute phase response and coagulation/fibrinolysis in chronic spontaneous urticaria. Allergy Asthma Clin Immunol. 2018;14:27. pmid:30026764

12. Kolkhir P, Pogorelov D, Olisova O. CRP, D-dimer, fibrinogen and ESR as predictive markers of response to standard doses of levocetirizine in patients with chronic spontaneous urticaria. Eur Ann Allergy Clin Immunol. 2017;49(4):189–92. pmid:28752724

13. Sánchez-Borges M, Capriles-Hulett A, Caballero-Fonseca F, González-Aveledo L. Biomarkers of treatment efficacy in patients with chronic spontaneous urticaria. Eur Ann Allergy Clin Immunol. 2018;50(1):5–9. pmid:29350016

14. Kay AB, Ying S, Ardelean E, Mlynek A, Kita H, Clark P, et al. Elevations in vascular markers and eosinophils in chronic spontaneous urticarial weals with low-level persistence in uninvolved skin. Br J Dermatol. 2014;171(3):505–11. pmid:24665899

15. Kolkhir P, Church MK, Altrichter S, Skov PS, Hawro T, Frischbutter S, et al. Eosinopenia, in Chronic Spontaneous Urticaria, Is Associated with High Disease Activity, Autoimmunity, and Poor Response to Treatment. J Allergy Clin Immunol Pract. 2019.

16. Altrichter S, Peter HJ, Pisarevskaja D, Metz M, Martus P, Maurer M. IgE mediated autoallergy against thyroid peroxidase—a novel pathomechanism of chronic spontaneous urticaria? PLoS One. 2011;6(4):e14794. pmid:21532759

17. Sánchez J, Zakzuk J, Cardona R. Evaluation of a Guidelines-Based Approach to the Treatment of Chronic Spontaneous Urticaria. J Allergy Clin Immunol Pract. 2018;6(1):177–82.e1. pmid:28709817

18. Sánchez Borges M, Tassinari S, Flores A. [Epidemiologic features in patients with antihistamine-resistant chronic urticaria]. Rev Alerg Mex. 2015;62(4):279–86. pmid:26556663

19. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605. pmid:19477892

20. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: Developing a prognostic model. BMJ. 2009;338:b604. pmid:19336487

21. Heus P, Damen JAAG, Pajouheshnia R, Scholten RJPM, Reitsma JB, Collins GS, et al. Uniformity in measuring adherence to reporting guidelines: the example of TRIPOD for assessing completeness of reporting of prediction model studies. BMJ Open. 2019;9(4):e025611. pmid:31023756

22. Heus P, Damen JAAG, Pajouheshnia R, Scholten RJPM, Reitsma JB, Collins GS, et al. Poor reporting of multivariable prediction model studies: towards a targeted implementation strategy of the TRIPOD statement. BMC Med. 2018;16(1):120. pmid:30021577

23. Collins G, Reitsma J, Altman D, Moons K. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594. pmid:25569120

24. Moons K, Altman D, Reitsma J, Loannidis J, Macaskill P, Steyerberg E, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162 (1):W1–73. pmid:25560730

25. Sánchez J, Sánchez A, Cardona R. Causal Relationship Between Anti-TPO IgE and Chronic Urticaria by. Allergy Asthma Immunol Res. 2019;11(1):29–42. pmid:30479075

26. Lee SJ, Ha EK, Jee HM, Lee KS, Lee SW, Kim MA, et al. Prevalence and Risk Factors of Urticaria With a Focus on Chronic Urticaria in Children. Allergy Asthma Immunol Res. 2017;9(3):212–9. pmid:28293927

27. Zuberbier T, Aberer W, Asero R, Abdul Latiff AH, Baker D, Ballmer-Weber B, et al. The EAACI/GA2LEN/EDF/WAO Guideline for the Definition, Classification, Diagnosis and Management of Urticaria. The 2017 Revision and Update. Allergy. 2018.

28. Kolkhir P, André F, Church MK, Maurer M, Metz M. Potential blood biomarkers in chronic spontaneous urticaria. Clin Exp Allergy. 2017;47(1):19–36. pmid:27926978

29. Church MK, Kolkhir P, Metz M, Maurer M. The role and relevance of mast cells in urticaria. Immunol Rev. 2018;282(1):232–47. pmid:29431202

30. Kolkhir P, Borzova E, Grattan C, Asero R, Pogorelov D, Maurer M. Autoimmune comorbidity in chronic spontaneous urticaria: A systematic review. Autoimmun Rev. 2017.

31. Kolkhir P, Metz M, Altrichter S, Maurer M. Comorbidity of chronic spontaneous urticaria and autoimmune thyroid diseases: A systematic review. Allergy. 2017;72(10):1440–60. pmid:28407273

32. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49(12):1373–9. pmid:8970487

33. Riley RD, Ensor J, Snell KIE, Harrell FE, Martin GP, Reitsma JB, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441. pmid:32188600

34. Kulthanan K, Chaweekulrat P, Komoltri C, Hunnangkul S, Tuchinda P, Chularojanamontri L, et al. Cyclosporine for Chronic Spontaneous Urticaria: A Meta-Analysis and Systematic Review. J Allergy Clin Immunol Pract. 2018;6(2):586–99. pmid:28916431

35. Dressler C, Werner RN, Eisert L, Zuberbier T, Nast A, Maurer M. Chronic inducible urticaria: A systematic review of treatment options. J Allergy Clin Immunol. 2018;141(5):1726–34. pmid:29438771

36. Maurer M, Metz M, Brehler R, Hillen U, Jakob T, Mahler V, et al. Omalizumab treatment in patients with chronic inducible urticaria: A systematic review of published evidence. J Allergy Clin Immunol. 2018;141(2):638–49. pmid:28751232

37. Guillen-Aguinaga S, Jauregui Presa I, Aguinaga-Ontoso E, Guillen-Grima F, Ferrer M. Updosing nonsedating antihistamines in patients with chronic spontaneous urticaria: a systematic review and meta-analysis. Br J Dermatol. 2016;175(6):1153–65. pmid:27237730

38. Sharma M, Bennett C, Carter B, Cohen SN. H1-antihistamines for chronic spontaneous urticaria: an abridged Cochrane Systematic Review. J Am Acad Dermatol. 2015;73(4):710–6.e4. pmid:26253363

39. Núñez E, Steyerberg EW, Núñez J. [Regression modeling strategies]. Rev Esp Cardiol. 2011;64(6):501–7. pmid:21531065

40. Hernández-Avila M, Garrido F, Salazar-Martínez E. Sesgos en estudios epidemiológicos Salud pública de méxico. 2000:438–46.

41. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. pmid:23393430

42. Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, et al. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10(2):e1001380. pmid:23393429

43. Steyerberg EW, Borsboom GJ, van Houwelingen HC, Eijkemans MJ, Habbema JD. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004;23(16):2567–86. pmid:15287085