1. Introduction
Risk management represents the main instrument through which Healthcare Institutions can supervise healthcare services, as well as improve their efficiency [1,2], in order to develop targeted improvement strategies and invest available resources in a proficient way [3,4,5].
Historically, the concept of implementing quality has often been hampered by the lack of objective, measurable, and comparable data: in this context, information tools, generically called indicators, can help administrators and health professionals in highlighting the criticalities of the system to direct priority choices [6].
Over the last few years, initiatives aimed at advertising rankings of hospitals or individual professionals have spread [7,8], defined according to the clinical results obtained, and documented through indicators [9,10,11]. In the current Italian health system [12,13], accreditation of health facilities is an indispensable prerequisite for any health facility to become an effective provider of remunerated services [14]. Accreditation, as configured by national legislation, represents a qualified tool for selecting providers on behalf of the National Health Service, characterized by the necessary correspondence to a series of requirements [15]. These are directly related to expected quality levels, as well as to the temporariness of their adequacy recognition. This system aims to promote the process of continuous improvement of performance quality, organization efficiency, and use of resources and training [16,17].
As shown in the literature [18,19,20,21,22,23], individual hospitals that adopted such initiatives have developed greater attention to the quality of care, or rather, to those aspects of care that were specifically the subject of public evaluation. “Quality of care” refers to a collection of different dimensions related to various aspects of the work of both healthcare professionals and organizations [22,24,25,26,27,28,29], defined by a criteria of accessibility, continuity, effectiveness, efficiency, clinical and organizational appropriateness, safety [27], timeliness, centrality of patients [28], and human resources development [22,29].
Scientific evidence identifies indicators as the best tool to measure performance values [30,31]. Defined as “a synthetic measure, generally expressed in quantitative form, coinciding with a variable or composed of several variables, capable of assuming the trend of the phenomenon to which it refers” [32], these variables are characterized by high information content, capability to allow a rapid evaluation of phenomena that are sometimes very complex, and to provide pivotal elements in operational decisions; furthermore, they can be used to compare phenomena over time (at different moments) and in space (in different realities), or concerning a goal that is to be achieved or maintained.
It can, therefore, be quantitative, ordinal, or semi-quantitative or -qualitative. To establish a set of indicators, the structure in which they are inserted, expressing the value of a specific characteristic, should be clarified. Homogeneous groups of indicators are included within the same index, which is a more complex tool in which multiple components (indicators) can interact with each other in different ways, through one or more mathematical operators. The index provides an easier parameter, which is more immediate to understand. Similarly, indices can be included in a macro-group called “dimension”, referring to that thematic area that aggregates performance indicators about the homogeneity of the phenomena they intend to measure and evaluate [33]. Within dimensions, sub-dimensions are provided to simplify the analysis of investigated phenomena.
To achieve effective and efficient use of indicators, however, it is necessary to constantly remember their limits. In fact, it is not possible to affirm anomalies or important variations of a phenomenon based only on the indicator’s alteration compared to its reference value (standard). At least a certain number of elements must be anomalous to correctly identify, with an acceptable degree of certainty, a real alteration in processes or phenomena. Therefore, the construction of good-quality indicators is fundamental for both their validity and use. To guarantee indicators’ completeness, validity, and precision, controls should be programmed, monitoring both archived data and newly collected ones [34].
For an indicator to provide quality of care information, it must be compared, either with values considered as reference (standard), or with values of the same unit under examination obtained at different times, or with those of other observation units [35]. Standards, depending on the role they play, can be summarized as follows: starting standards (at the beginning of the monitoring activity), improvement standards (objective to be achieved), quality standards (the best possible quality), and accreditation standards (mandatory levels for the acquisition of specific authorizations).
Three main uses of indicators can be distinguished [36,37]:
Scientific and research purposes;
Internal evaluation and improvement within an organization (continuous quality improvement and internal evaluation of healthcare organizations);
Evaluation and improvement of an organization in relation to the outside world (national indicators). This makes it possible to compare similar organizations (benchmarking), and is also widely applied during accountability and accreditation processes [38,39,40].
In this evolutionary context, healthcare-associated performance indicators constitute a powerful tool, through which decision-makers can grasp initial conditions of a system, identify quality and safety issues, and quantify reasonably achievable objectives [41,42,43,44]. Moreover, indicators allow to verify the correspondence between results obtained and those expected, and to measure the impact of carried-out activities [45]. A further objective of performance indicators analysis would be to hinder the phenomenon of defensive medicine that affects many specialist fields of medicine [46]. It is important to underline that monitoring actions are not always easy from a technical-methodological point of view [47,48]. Healthcare is a multidimensional product, and there are no stable relationships, either between overall healthcare expenditure and resources supply, nor between resources and healthcare services (due to management efficiency diversity), nor, finally, between services and health outcomes [49].
Therefore, it is necessary to focus on one or more sets of indicators capable of providing information regarding a plurality of phenomena ranging from allocation of physical resources to expenditure [50]. The construction of this integrated and balanced system of indicators presents further difficulties. The plurality of recipients determines multiple interests, which are not necessarily convergent. Type and quality of produced information, ways of presenting data, level of complexity, and disaggregation adopted in the process of producing indicators may, therefore, differ, depending on the user to whom the selected system of indicators is addressed.
Finally, emphasis on healthcare assessment systems is progressively shifting from simple spatial-temporal comparisons to more complex aspects related to benchmarking, extending to the clinical-epidemiological field [51].
Traditionally, the analysis of health services and their quality is based on Donabedian’s conceptual model, which provides three indicator topologies [52,53,54,55,56]:
Structure indicators;
Process indicators;
Outcome indicators.
Process measures are essential in providing guidance to professionals on how to modify a path to improve its impact on patient care outcomes [57,58].
Notably, outcome indicators document a change in clinical (health, mortality, morbidity), economic (direct and indirect costs), and humanistic (quality of life, user satisfaction) care standards, and, furthermore, they tend to directly highlight results and are easier to understand [59,60].
Nonetheless, a widely stressed concept is that an adequate use of indicators lies in their actionability, which can be summarized in the need to be simultaneously “fit for purpose” and “fit for use” [61].
Therefore, the first goal of our analysis was to develop a system for assessing and monitoring the quality of services provided by 3 large Italian health facilities. The application of these concepts to the clinical and managerial reality of large healthcare facilities is a novelty in the scientific panorama, as it has not been possible to find articles similar to the present in the literature.
Therefore, the selection of categories of indicators that maximize the performance improvement process was based on the analysis of the 6 dimensions of quality: safe, effective, patient-centered, timely, efficient, and equitable care [62]. On the basis of these requirements, the proposed data represent an absolute novelty in the clinical and managerial landscape of healthcare facilities; moreover, in addition to the “photographic” value in their ability to objectively show a present scenario, they possess the potential for a future prospective analysis, and for a possible comparison between international structures, in order to assist a wide-range improvement process on issues of absolute relevance (e.g., fight against antimicrobial resistance).
2. Materials and Methods
A set of performance indicators was developed based on data from the medical-legal claims of Umberto I General Hospital, Agostino Gemelli University Hospital Foundation and Campus Bio-Medico University Hospital Foundation, from 1 January 2013 to 31 December 2020. Starting from July 2018, a unification and normalization operation of existing reports, relating to the management of civil litigation operated by the Legal Medicine Units of Umberto I General Hospital, Agostino Gemelli University Hospital Foundation and the Department of Clinical Affairs of Campus Bio-Medico University Hospital Foundation, was carried out.
Umberto I General Hospital is a mixed welfare and educational institution, equipped with a second-level Emergency Department and 1235 ordinary beds. In the last year of operation, it carried out about 41,000 hospitalizations, while more than 140,000 patients accessed the emergency room. It also provides 1,000,000 services per year between instrumental diagnostics and medical examinations, 900,000 laboratory tests, 20,000 histopathological investigations, and as many radiological investigations.
Agostino Gemelli University Hospital Foundation is both a welfare and an educational institution. Eight clinical and research departments involve 113 medical care units, including 86 complex operating units, 27 simple operating areas, and 1536 beds. In one year, 215 organ transplantations were performed, 94,509 patients were discharged, and 83,419 patients accessed the emergency room.
Campus Bio-Medico University Hospital Foundation is a mixed welfare and educational institution, equipped with an Emergency Department, 296 ordinary beds, and more than 60 medical care units. In the last year of operation, it carried out about 28,700 hospitalizations and 530,000 outpatient treatments, while more than 11,600 patients accessed the emergency room (Table 1).
The collected material was initially heterogeneous in content and structure, and provided for direct consultation of 2098 files. All cases included in the database presented the following features:
they were all categorized according to the International Classification for Patient Safety (ICPS) system [63], with particular attention to the following parameters: age, gender, unit involved, event date and type, outcome (for patient and structure involved);
economic quantification of the request, technical opinion, risk of loss, eventual amount paid;
chronological specification of the judicial (or extrajudicial) phase.
Later, the most significant data from the collected cluster of clinical records were extrapolated. In our study, the high number of cases ensures a high level of representativeness compared to the reference population. All data and information acquired were collected and organized digitally on a Microsoft Excel single sheet, through an entry string structured according to descriptive indicators of event and damage, or presumed damage, of subject, as well as parameters of medico-legal evaluation.
Following the first phase of database homogenization obtained through individual review of cases in close collaboration with legal offices, Excel calculation functions were applied through the construction of “pivot” tables.
Finally, results were summarized in graphical form and using summary tables. Collected data were distributed according to a wide qualitative and quantitative variability, especially about opening date of the claim and corresponding event, which is why our reference sample may be slightly inconstant depending on the case, which will be specified from time to time.
Although this study was conducted as multicentric, the ultimate aim is not the comparison between healthcare-related litigation data from the structures involved. Therefore, the data presented will be exposed by combining the individual values found, thus considering the structures in question as a single group.
2.1. Indicators Set
The set of indicators was chosen to evaluate health facilities’ performance. The choice was made possible by the application of the Process Analysis Method (PAM) [64], which demonstrated its suitability for performance evaluation of Umberto I General Hospital in a previous experience by Scopetti et al. [65]. The implementation of the PAM system consists of five phases:
Phase I—Overview of the claims. During this phase, all files were examined;
Phase II—Definition of performance in claims management, established as the contribution in terms of results of a subject to the achievement of an objective;
Phase III—Setting system limits. The spatial criterion consisted of all claims based on civil and health liability, while the temporal one was the limitation of claims to the period 2013–2020;
Phase IV—Setting up the performance scoreboard. In this phase, a set of indicators was selected and standardized for name, definition, and calculation formula. Notably, all indicators were validated against internationally recognized benchmarks, such as the Agency for Healthcare Research and Quality (AHRQ);
Phase V—Verification by reviewing all data, measures, and indicators.
Moreover, since the aim of this study was to ensure an improvement in quality of care, the choice of indicators was guided by their scientific validity and, as well as their measurability.
In particular, the specific dimension identified was “clinical setting”. In this dimension, the index used was “quality of care”. All indicators were chosen on the basis of the most representative claim-related events that occurred within the selected structures, and were expressed in the form of rates (ratio between two numerical quantities) (Table 2).
In contrast to what was done previously [65], no internal standard was applied to the analysis of the identified indicators. Internal standard refers to the ratio between average of claims reported per year and the cumulative number of annual hospitalizations [64]. Such an instrument is particularly useful to monitor the trend of standardized variables over time, but it reaches maximum significance in the event that the data come from a homogeneous source, and not from multiple structures. In fact, their variability would prevent a value from being correctly evaluated based on a pre-set threshold dependent on the values obtained. Since the objective of this paper is to observe the trend of claims on a larger scale than the individual structure, the use of KPIs alone can, therefore, be considered sufficient.
2.2. Bias Risk Assessment
Since the present analysis constitutes the first attempt to start a line of applied research on the use of performance indicators in a healthcare setting, an approach based on eliminating the major complexities related to data extraction has been adopted, as well as on the use of a numerical and graphic language based on simplicity and immediate understanding of the results shown. The use of data relating to the outcome of health services interfaces perfectly with this purpose, but exposes the analysis conducted to the risk of not considering equally important elements, including the compliance of companies with structural and accreditation requirements, or of not deepening the adequacy of care and treatment pathways. These aspects, extremely heterogeneous from each other, require a dedicated disquisition.
Furthermore, the events considered in this analysis consist of the claims reported in the period of interest, and this entails a limited view of the events resulting from legal disputes; the number of claims does not necessarily correspond to the total number of adverse events, which, however, can be ignored for numerous reasons, including failure to report to the health management in the form of an audit.
The multicentric approach adopted in this paper allows, in any case, to limit several critical elements determined by the influence that the structural and welfare elements of each structure have in terms of outcome; moreover, from a management point of view, based on the analysis of clinical efficiency and cost effectiveness, the monitoring of the trend of claims certainly constitutes an optimal approach to understanding the degree of quality perceived by patients and the related economic aspect.
3. Results
After collecting, analyzing, and subsequently indexing all files, a preliminary consideration is that a slight fluctuation in the incidence of the single categories of events identified was observed among the structures investigated, but it should be emphasized that, as previously stated, the comparison between the single centers was not an objective of this paper, which, on the contrary, was based on the analysis of a database as broad and representative as possible of the Italian national scenario. However, the work of merging the data, and careful observation of the same, revealed characteristics of homogeneity, so, the extrapolated considerations can be considered free from “disequilibrium” biases.
From the total number, only 779 records (37.1%) were attributable to the categories indexable for the present analysis. Obtained results are represented and summarized in the following graphic representation (Figure 1).
It can be preliminarily stated that, as shown in the graphic above, the type of event most represented within the structures considered is healthcare-related accidental fall (21%), with a slightly higher incidence than healthcare-associated infections (HAI, 20%). Below, the two most represented categories after the previous ones are that of surgical site infections (SSI) and wrong-site surgery or wrong procedure (14% of incidence for both). These data partially follow what was observed in a previous experience of litigation data, relating only to the Umberto I General Hospital, underlining the high incidence of events related to HAIs and the impact of the surgical sphere on the total number of claims [66]. The events characterized by the lowest percentage of incidence were the acts of violence against healthcare workers (1%), suicides in medical setting, and cases of maternal decease after birth (both with an incidence of less than 1%).
An analysis of the temporal trend of the events analyzed made it essential to standardize the total number of claims per year (Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11). This made it possible to observe a consistent deflationary trend of claims related to accidental falls in healthcare settings and transfusion-related adverse effects, which reflects adequate organizational and procedural management of these events by examined healthcare facilities. This result takes on fundamental importance when compared with the absolute number of events, which, instead, shows an increasing trend over time: the importance of an adequate KPI, in this case, lies precisely in its ability to highlight how the number of specific adverse events compared to the total show a decreasing trend as an index of increase in the quality of care, with respect to the chosen item.
On the other hand, the SSIs are maintaining a stable trend during the selected period (Figure 4), highlighting the possibility of further preventive interventions. In addition, the number of HAIs over time appears to have increased, highlighting the need for more stringent measures against an entity that is assuming the character of a global emergency. Even an event of a psychiatric nature, such as suicide, has been the subject of our study through the elaboration of an appropriate indicator.
Finally, it is necessary to specify that no further investigations of a technical-statistical nature have been carried out about the three less-represented categories, since the numbers have not reached sufficiency to allow any statistically significant estimates (Figure 12).
However, it is significant to note how acts of violence against healthcare personnel saw a peak incidence in 2013 (3 episodes), followed by two years without such events, and, subsequently, by single episodes recorded per year; despite the smallness of the numbers presented, the reversal of the initial trend can still be interpreted positively in terms of adaptation of the structures to the needs of the patients, in terms of organization, communication, and quality of the service offered. On the other hand, the trend of suicides and the single episode of maternal death after birth can also be considered as anecdotal episodes and are, as such, worthy of further study, but without the need to proceed with further analytical disquisitions.
4. Discussion
Our study moves from a methodological approach, strongly oriented to scientific and doctrinal principles of academic forensic medicine within hospital organizational and professional systems, to manage health liability litigation [66]. The ethical dimension of clinical risk management was also considered in the choice of a multicentric setting that starts from shared experience to seek scientific truth, according to the ethics of a “Job Well Done” [67].
Moreover, an accurate analysis of litigation may help hospital systems to make process indicators, for example, extending the application of predictive tools widely used in the insurance world to the evaluation of incident reporting [68].
The objective behind the study was twofold. First, information was collected and organized for internal company reporting, regarding civil litigation that has been affecting the hospital, to relate results obtained with the phenomenon of medical malpractice on a national scale. Secondly, performance indicators were produced, allowing to carry out a standardized assessment of the clinical dimension of the structure. Even the production of an indicator for the suicide event is indicative of how important it is to assess the quality of a healthcare structure that requires personalized medical interventions for the patient [69,70,71].
Therefore, the present analysis highlights how, following a correct and rigorous categorization of hospital events, it is possible to analyze these medico-legal aspects using a small number of indicators.
On the other hand, it is important to consider how a non-negligible percentage of events on the total (1319 out of 2098 selected files) was difficult to index, being constituted of events such as theft or damage to property. These events certainly play a marginal role within the present discussion, since they are phenomena not directly linked to the treatment process, and are, therefore, of poor scientific interest in the present context. Furthermore, since they mainly depended on the structural and managerial set-up of the individual structures, these aspects are highly uneven among the analyzed centers. Despite this, they can still be categorized and analyzed for purposes other than that of the present investigation, so these aspects will be examined separately.
Among specific scientific literature, many studies apply process indicators in the context of performance assessments. The organizational and decision-making process that follows compares results with desirable values for each indicator, thus leading to corrective actions. As a result, those parameters that were found to be non-compliant are modified and implemented. In the meantime, company performance improvement programs are defined and priorities and margins for improvement identified.
However, these indicators, while potentially able to predict an improvement in care outcomes, do not provide information on the outcomes of care (outcomes). For this reason, we propose a set of 13 outcome indicators in the assessment of “quality of care”, which do not need a comparative standard for a longitudinal analysis.
Since there are no previous studies evaluating outcome indicators in a clinical setting, we suggest a set of 13 indicators. To address the clinical performance of the analyzed healthcare facility, our study focused on “quality of care”, and, therefore, on those outcome indicators related to this specific index. The elaboration process for all the indicators we chose moves from a thorough analysis of the most recurrent claims in the healthcare setting to their economic impact on the healthcare structure, and, further, to the evidence of possible corrective actions to reduce their frequency.
Several results that have emerged are highly encouraging. The first is certainly constituted by the homogeneity found in the classes of indicators identified among the structures being analyzed, which allows us to hypothesize the possibility of using these indicators on a large scale (e.g., city, province, region, nation, continent) in order to build a surveillance network of adverse clinical events that allows monitoring the trend of phenomena of great interest, including HAIs. Unfortunately, the decision to use KPIs based on the outcome requires a critical filter on the emerged data, including the degree of awareness of citizens towards specific health-related phenomena that also influence the reporting rate [65]. If the decision to focus the analysis on the rate of claims may lead to limitations, it is also possible to state that this approach provides the basis for an economic assessment of the burden represented by the single items of damage. This could allow clinical risk management decisions to be oriented on the basis of the economic expenditure linked to individual categories; in the future, however, the application of the same methodological criteria for the construction of process KPIs could represent a valuable tool with which to close the loop relating to the improvement of the quality of care.
In any case, the main criticality linked to the application of such a tool to other healthcare facilities is the possible lack of a precise register that contains information on all claims based on time, operating unit, category of claim, the characteristics of the patient, and news of an economic nature linked to the judicial progress of the claim. From this point of view, a valuable tool has been provided in Italy by the provisions of Law no. 24/2017, according to which all healthcare facilities are required to prepare and publish online all litigation data per year [72]; in this way, the easy availability of sensitive data relating to the legal expenses incurred by the structures can be an ally for the processing of such data by specially trained external bodies.
Therefore, our results provide a picture of the “quality of care” delivered by Umberto I General Hospital, Campus Bio-Medico University Hospital Foundation and Agostino Gemelli IRCCS University Hospital Foundation, from 2013 to 2020. Obtained data could be used to compare our hospital performance over time and with different hospitals. In fact, while the formulation of process indicators moves from guidelines analysis and standards identification, outcome indicators are not expected to fulfill any previously defined requirement, simply providing an analytic tool for performance evaluation [72].
Moreover, a limitation of the present analysis is constituted by the lack of consideration of fundamental aspects related to the hospital management of claims, as the economic and/or humanistic point of view; further analyses will be able to demonstrate the definitive suitability of the indicators proposed in this paper for practical use in the hospital setting.
5. Conclusions
In this study, the outcome indicators were conceived starting from a retrospective analysis of the medico-legal claims data. Therefore, the study limited itself to portraying the adverse events that resulted in a request for compensation and/or the initiation of legal proceedings. However, we are convinced that this is only a first step. The science of safety in care must aim for professional excellence. It is necessary to go further and identify standardized indices that can reveal adverse events, even if there have been no claims for compensation or the initiation of legal proceedings or a spontaneous report by the health professionals.
As previously stated, Clinical Risk Management represents one of the structural programmatic activities of the healthcare system. The need to identify more suitable tools for a “real” evaluation of different healthcare settings, as well as any possibility for improvement, has led to a series of research strategies that, comparing different organizations, would provide necessary elements to define policies of quality improvement. In this context, the use of indicators in both public and private healthcare realities is destined to become an essential asset for every hospital. Precisely, the practice of health facilities accreditation, as provided for by the National Health Plan in the last decade, responds to the need to select good providers of health services based on qualitative criteria.
This will allow health organizations to effectively monitor verified and unreported adverse events, and allow the possibility of implementing measures and interventions aimed at the safety of medical care and procedures to protect the patient.
Conceptualization, N.D.F. and M.S.; methodology, N.D.F.; software, M.S.; validation, R.L.R., G.V. and F.D.M.; formal analysis, G.D.; investigation, V.T.; resources, G.V. and V.F.; data curation, M.S. and F.F.; writing—original draft preparation, N.D.F. and G.D.; writing—review and editing, V.F.; visualization, V.M.G. and A.D.B.; supervision P.F.; project administration, R.R.; funding acquisition, N.D.F. All authors have read and agreed to the published version of the manuscript.
Not applicable.
Not applicable.
Not available.
The authors declare no conflict of interest.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 1. “Quality of care” analysis of Umberto I General Hospital, Campus Bio-Medico University Hospital Foundation and Agostino Gemelli IRCCS University Hospital Foundation, from 2007 to 2020.
Enlarge this image.Figure 12. Maternal decease after birth rate, violence against healthcare workers, suicide in medical setting rate.
Briefly reported numbers of the selected structures.
| Structure | Umberto I General Hospital | Agostino Gemelli University Hospital Foundation | Campus Bio-Medico University Hospital Foundation |
|---|---|---|---|
| Ordinary beds (no.) | 1235 | 1536 | 296 |
| No. of hospitalizations in the last year | 41,000 | 94,509 | 28,700 hospitalizations |
| No. of accesses to the emergency room | Over 140,000 | 83,419 | 11,600 |
Ideal characteristics of an indicator, according to specific categories.
| Dimension | Index | Outcome indicator | Description |
|---|---|---|---|
| Clinical setting | Quality of care | Accidental falls rate | Number of accidental falls/total of claims |
| Healthcare-related infections rate | Number of healthcare-related infections/total of claims | ||
| Surgical site infections rate | Number of surgical site infections/total of claims | ||
| Transfusion-related adverse events rate | Number of transfusions-related adverse events/total of claims | ||
| Drug administration-related adverse events rate | Number of drug administration-related adverse events/total of claims | ||
| Birth injuries rate | Number of maternal birth injuries/total of claims | ||
| Maternal deceases after birth rate | Number of maternal deceases after birth/total of claims | ||
| Wrong-site surgery/wrong procedure rate | Number of wrong procedures and wrong-site surgeries/total of claims | ||
| Retained surgical instruments’ rate | Number of cases of retained surgical instruments/total of claims | ||
| Clinical devices malfunction rate | Number of cases related to clinical devices malfunction/total of claims | ||
| Triage error rate | Number of cases related to triage errors/total of claims | ||
| Violence against healthcare workers rate | Number of cases related to violence against healthcare workers/total of claims | ||
| Suicide in medical setting rate | Number of suicides in medical setting/total of claims |
References
1. Pollitt, C.; Harrison, S.; Dowswell, G.; Jerak-Zuiderent, S.; Bal, R. Performance regimes in health care: Institutions, critical junctures and the logic of escalation in England and the Netherlands. Evaluation; 2010; 16, pp. 13-29. [DOI: https://dx.doi.org/10.1177/1356389009350026]
2. Del Fante, Z.; Di Fazio, N.; Papale, A.; Tomao, P.; Del Duca, F.; Frati, P.; Fineschi, V. Evaluation of Physical Risk during Necropsy and Morgue Activities as Risk Management Strategy. Int. J. Environ. Res. Public Health; 2021; 18, 8266. [DOI: https://dx.doi.org/10.3390/ijerph18168266]
3. Carrin, G.; James, C. Key performance indicators for the Implementation of social health insurance. Appl. Health Econ. Health Policy; 2005; 4, pp. 15-22. [DOI: https://dx.doi.org/10.2165/00148365-200504010-00004] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16076235]
4. Ferracuti, S.; Barchielli, B.; Napoli, C.; Fineschi, V.; Mandarelli, G. Evaluation of official procedures for suicide prevention in hospital from a forensic psychiatric and a risk management perspective. Int. J. Psychiatry Clin. Pract.; 2020; 24, pp. 245-249. [DOI: https://dx.doi.org/10.1080/13651501.2020.1759647]
5. Young, M.; Smith, M.A. Standards and evaluation of healthcare quality, safety, and person centered care. StatPearls [Internet]; StatPearls Publishing: Treasure Island, FL, USA, 2022.
6. Sisk, R.; Lin, L.; Sperrin, M.; Barrett, J.K.; Tom, B.; Diaz-Ordaz, K.; Martin, G.P. Informative presence and observation in routine health data: A review of methodology for clinical risk prediction. J. Am. Med. Inf. Assoc.; 2021; 28, pp. 155-166. [DOI: https://dx.doi.org/10.1093/jamia/ocaa242]
7. Guiñón, L.; Soler, A.; Gisell Díaz, M.; Fernández, R.M.; Rico, N.; Bedini, J.L.; Mira, A.; Alvarez, L. Analytical performance assessment and improvement by means of the Failure mode and effect analysis (FMEA). Biochem. Med.; 2020; 30, pp. 250-256. [DOI: https://dx.doi.org/10.11613/BM.2020.020703] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32292281]
8. Braithwaite, J.; Hibbert, P.; Blakely, B.; Plumb, J.; Hannaford, N.; Long, J.C.; Marks, D. Health system frameworks and performance indicators in eight countries: A comparative international analysis. SAGE Open Med.; 2017; 5, 2050312116686516. [DOI: https://dx.doi.org/10.1177/2050312116686516]
9. Allen, T.; Walshe, K.; Proudlove, N.; Sutton, M. Do performance indicators predict regulator ratings of healthcare providers? Cross-sectional study of acute hospitals in England. Int. J. Qual. Health Care; 2020; 32, pp. 113-119. [DOI: https://dx.doi.org/10.1093/intqhc/mzz101]
10. Mainz, J. Defining and classifying clinical indicators for quality improvement. Int. J. Qual. Health Care; 2003; 15, pp. 523-530. [DOI: https://dx.doi.org/10.1093/intqhc/mzg081]
11. Beaussier, A.-L.; Demeritt, D.; Griffiths, A.; Rothstein, H. Steering by their own lights: Why regulators across Europe use different indicators to measure healthcare quality. Health Policy; 2020; 124, pp. 501-510. [DOI: https://dx.doi.org/10.1016/j.healthpol.2020.02.012]
12. Berg, M.; Meijerink, Y.; Gras, M.; Goossensen, A.; Schellekens, W.; Haeck, J.; Kingma, H. Feasibility first: Developing public performance indicators on patient safety and clinical effectiveness for Dutch hospitals. Health Policy; 2005; 75, pp. 59-73. [DOI: https://dx.doi.org/10.1016/j.healthpol.2005.02.007] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16298229]
13. Albolino, S.; Bellandi, T.; Cappelletti, S.; Di Paolo, M.; Fineschi, V.; Frati, P.; Turillazzi, E. New Rules on Patient’s Safety and Professional Liability for the Italian Health Service. Curr. Pharm. Biotechnol.; 2019; 20, pp. 615-624. [DOI: https://dx.doi.org/10.2174/1389201020666190408094016] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30961486]
14. Mossialos, E.; Wenzl, M.; Osborn, R.; Sarnak, D. International profiles of health care systems. CADTH; 2015; pp. 97-106. [DOI: https://dx.doi.org/10.15868/socialsector.25100]
15. Mohammadfam, I.; Kamalinia, M.; Momeni, M.; Golmohammadi, R.; Hamidi, Y.; Soltanian, A. Evaluation of the quality of occupational health and safety management systems based on key performance indicators in certified organizations. Saf. Health Work.; 2017; 8, pp. 156-161. [DOI: https://dx.doi.org/10.1016/j.shaw.2016.09.001] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28593071]
16. Agrizzi, D.; Jaafaripooyan, E.; Akbarihaghighi, F. Key Performance Indicators (KPIs) for Healthcare Accreditation System. 2010; Available online: https://eprints.soton.ac.uk/79297/1/CRAAG-10-01.pdf (accessed on 4 May 2022).
17. Murphy, A.; Wakai, A.; Walsh, C.; Cummins, F.; O’Sullivan, R. Development of key performance indicators for prehospital emergency care. Emerg. Med.; 2016; 33, pp. 286-292. [DOI: https://dx.doi.org/10.1136/emermed-2015-204793] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26796739]
18. D’errico, S.; Pennelli, S.; Colasurdo, A.P.; Frati, P.; Sicuro, L.; Fineschi, V. The right to be informed and fear of disclosure: Sustainability of a full error disclosure policy at an Italian cancer centre/clinic. BMC Health Serv. Res.; 2015; 15, 130. [DOI: https://dx.doi.org/10.1186/s12913-015-0794-3]
19. Azami-Aghdash, S.; Tabrizi, J.S.; Sadeghi-Bazargani, H.; Hajebrahimi, S.; Naghavi-Behzad, M. Developing performance indicators for clinical governance in dimensions of risk management and clinical effectiveness. Int. J. Qual. Health Care; 2015; 27, pp. 110-116. [DOI: https://dx.doi.org/10.1093/intqhc/mzu102]
20. Kelley, E.; Hurst, J. Health Care Quality Indicators Project: Conceptual Framework Paper; OECD Health Working Papers: Paris, France, 2006.
21. El Sayed, M.J. Measuring quality in emergency medical services: A review of clinical performance indicators. Emerg. Med. Int.; 2011; 2012, 161630. [DOI: https://dx.doi.org/10.1155/2012/161630]
22. Gu, X.; Itoh, K. Performance indicators: Healthcare professionals’ views. Int. J. Health Care Qual. Assur.; 2016; 29, pp. 801-815. [DOI: https://dx.doi.org/10.1108/IJHCQA-12-2015-0142]
23. Nuti, S.; Seghieri, C.; Vainieri, M. Assessing the effectiveness of a performance evaluation system in the public health care sector: Some novel evidence from the Tuscany region experience. J. Manag. Gov.; 2013; 17, pp. 59-69. [DOI: https://dx.doi.org/10.1007/s10997-012-9218-5]
24. Khalifa, M.; Khalid, P. Developing strategic health care key performance indicators: A case study on a tertiary care hospital. Procedia Comput. Sci.; 2015; 63, pp. 459-466. [DOI: https://dx.doi.org/10.1016/j.procs.2015.08.368]
25. Botje, D.; Asbroek, G.T.; Plochg, T.; Anema, H.; Kringos, D.S.; Fischer, C.; Klazinga, N.S. Are performance indicators used for hospital quality management: A qualitative interview study amongst health professionals and quality managers in The Netherlands. BMC Health Serv. Res.; 2016; 16, 574. [DOI: https://dx.doi.org/10.1186/s12913-016-1826-3] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27733194]
26. Cinaroglu, S.; Baser, O. Understanding the relationship between effectiveness and outcome indicators to improve quality in healthcare. Total. Qual. Manag. Bus. Excel.; 2018; 29, pp. 1294-1311. [DOI: https://dx.doi.org/10.1080/14783363.2016.1253467]
27. Sheldon, T. Promoting health care quality: What role performance indicators?. Qual. Health Care; 1998; 7, pp. 45-50.
28. Sinelnikov, S.; Inouye, J.; Kerper, S. Using leading indicators to measure occupational health and safety performance. Saf. Sci.; 2015; 72, pp. 240-248. [DOI: https://dx.doi.org/10.1016/j.ssci.2014.09.010]
29. Magee, H.; Davis, L.J.; Coulter, A. Public views on healthcare performance indicators and patient choice. J. R. Soc. Med.; 2003; 96, pp. 338-342. [DOI: https://dx.doi.org/10.1177/014107680309600707]
30. Bein, M.A.; Unlucan, D.; Olowu, G.; Kalifa, W. Healthcare spending and health outcomes: Evidence from selected East African countries. Afr. Health Sci.; 2017; 17, pp. 247-254. [DOI: https://dx.doi.org/10.4314/ahs.v17i1.30]
31. Burlea-Schiopoiu, A.; Ferhati, K. The Managerial Implications of the Key Performance Indicators in Healthcare Sector: A Cluster Analysis. Healthcare; 2021; 9, 19. [DOI: https://dx.doi.org/10.3390/healthcare9010019]
32. Chae, Y.M.; Kim, H.S.; Tark, K.C.; Park, H.J.; Ho, S.H. Analysis of healthcare quality indicator using data mining and decision support system. Expert Syst. Appl.; 2003; 24, pp. 167-172. [DOI: https://dx.doi.org/10.1016/S0957-4174(02)00139-2]
33. NICE. Indicator Process Guide. 2019; Available online: https://www.nice.org.uk/media/default/Get-involved/Meetings-In-Public/indicator-advisory-committee/ioc-process-guide.pdf (accessed on 15 February 2023).
34. Madsen, M.; Kiuru, S.; Castrèn, M.; Kurland, L. The level of evidence for emergency department performance indicators: Systematic review. Eur. J. Emerg. Med.; 2015; 22, pp. 298-305. [DOI: https://dx.doi.org/10.1097/MEJ.0000000000000279]
35. Powell, A.; Davies, H.; Thompson, R. Using routine comparative data to assess quality of health care. Qual. Saf. Health Care; 2003; 12, pp. 122-128. [DOI: https://dx.doi.org/10.1136/qhc.12.2.122] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/12679509]
36. Rahimi, H.; Kavosi, Z.; Shojaei, P.; Kharazmi, E. Key performance indicators in hospital based on balanced scorecard model. J. Health Man. Inf.; 2017; 4, pp. 17-24.
37. Palmer, R.H. Quality health care. JAMA; 1996; 275, pp. 1851-1852. [DOI: https://dx.doi.org/10.1001/jama.1996.03530470079047] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/8642747]
38. Hofer, T.P.; Bernstein, S.J.; Hayward, R.A.; DeMonner, S. Validating Quality Indicators for Hospital Care. Jt. Comm. J. Qual. Patient Saf.; 1997; 23, pp. 455-467. [DOI: https://dx.doi.org/10.1016/S1070-3241(16)30332-7]
39. Campbell, S.M.; Braspenning, J.; Hutchinson, A.; Marshall, M. Research methods used in developing and applying quality indicators in primary care. Qual. Saf. Health Care; 2002; 11, pp. 358-364. [DOI: https://dx.doi.org/10.1136/qhc.11.4.358]
40. Magnan, S.; Solberg, L.I.; Giles, K.; Kottke, T.E.; Wheeler, J.W. Primary care, process improvement, and turmoil. J. Ambul. Care Manag.; 1997; 20, pp. 32-38. [DOI: https://dx.doi.org/10.1097/00004479-199710000-00006]
41. Arah, O.A.; Westert, G.P.; Hurst, J.; Klazinga, N.S. A conceptual framework for the OECD health care quality indicators project. Int. J. Qual. Health Care; 2006; 18, pp. 5-13. [DOI: https://dx.doi.org/10.1093/intqhc/mzl024]
42. Kelley, E.T.; Arispe, I.; Holmes, J. Beyond the initial indicators: Lessons from the OECD health care quality indicators project and the US national healthcare quality report. Int. J. Qual. Health Care; 2006; 18, pp. 45-51. [DOI: https://dx.doi.org/10.1093/intqhc/mzl027]
43. Fineschi, V. Healthcare-Associated Infections: Antibiotic Poly-therapies, Antibiotic Prophylaxis and Appropriate Policy for the Risk Management to Fight Adverse Events. Curr. Pharm. Biotechnol.; 2019; 20, pp. 606-608. [DOI: https://dx.doi.org/10.2174/138920102008190716152314]
44. Alsharf, I.A. Performance Management in Healthcare. J. Appl. Manag. Invest.; 2015; 4, pp. 45-52.
45. Giraldi, G.; Montesano, M.; Napoli, C.; Frati, P.; La Russa, R.; Santurro, A.; Scopetti, M.; Orsi, G.B. Healthcare-Associated Infections Due to Multidrug-Resistant Organisms: A Surveillance Study on Extra Hospital Stay and Direct Costs. Curr. Pharm. Biotechnol.; 2019; 20, pp. 643-652. [DOI: https://dx.doi.org/10.2174/1389201020666190408095811] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30961489]
46. Fineschi, V.; Arcangeli, M.; Di Fazio, N.; Del Fante, Z.; Fineschi, B.; Santoro, P.; Frati, P. Associazione Consulcesi Health And Onlus Futura Ricerca. Defensive Medicine in the Management of Cesarean Delivery: A Survey among Italian Physicians. Healthcare; 2021; 9, 1097. [DOI: https://dx.doi.org/10.3390/healthcare9091097]
47. Bankauskaite, V.; Dargent, G. Health systems performance indicators: Methodological issues. Presup. Y Gasto Publico; 2007; 49, pp. 125-137.
48. Scopetti, M.; Santurro, A.; Tartaglia, R.; Frati, P.; Fineschi, V. Expanding frontiers of risk management: Care safety in nursing home during COVID-19 pandemic. Int. J. Qual. Health Care; 2021; 33, mzaa085. [DOI: https://dx.doi.org/10.1093/intqhc/mzaa085] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32720688]
49. Van den Heuvel, J.; Niemeijer, G.C.; Does, R.J. Measuring healthcare quality: The challenges. Int. J. Health Care Qual. Assur.; 2013; 26, pp. 269-278. [DOI: https://dx.doi.org/10.1108/09526861311311454] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23729130]
50. Pourmohammadi, K.; Hatam, N.; Shojaei, P.; Bastani, P. A comprehensive map of the evidence on the performance evaluation indicators of public hospitals: A scoping study and best fit framework synthesis. Cost Eff. Resour. Alloc.; 2018; 16, 64. [DOI: https://dx.doi.org/10.1186/s12962-018-0166-z]
51. Bevan, G.; Evans, A.; Nuti, S. Reputations count: Why benchmarking performance is improving health care across the world. Health Econ. Policy Law; 2018; 14, pp. 141-161. [DOI: https://dx.doi.org/10.1017/S1744133117000561]
52. Van der Geer, E.; van Tuijl, H.F.; Rutte, C.G. Performance management in healthcare: Performance indicator development, task uncertainty, and types of performance indicators. Soc. Sci. Med.; 2009; 69, pp. 1523-1530. [DOI: https://dx.doi.org/10.1016/j.socscimed.2009.08.026]
53. Donabedian, A. An Introduction to Quality Assurance in Health Care; Oxford University Press: Oxford, UK, 2002.
54. Hejduková, P.; Kureková, L. National health systems’ performance: Evaluation WHO indicators. Procedia Soc. Behav. Sci.; 2016; 230, pp. 240-248. [DOI: https://dx.doi.org/10.1016/j.sbspro.2016.09.031]
55. Rademakers, J.; Delnoij, D.; de Boer, D. Structure, process or outcome: Which contributes most to patients’ overall assessment of healthcare quality?. BMJ Qual. Saf.; 2011; 20, pp. 326-331. [DOI: https://dx.doi.org/10.1136/bmjqs.2010.042358]
56. Donabedian, A. The quality of care: How can it be assessed?. JAMA; 1998; 260, pp. 1743-1748. [DOI: https://dx.doi.org/10.1001/jama.1988.03410120089033]
57. Podgórski, D. Measuring operational performance of OSH management system–A demonstration of AHP-based selection of leading key performance indicators. Saf. Sci.; 2015; 73, pp. 146-166. [DOI: https://dx.doi.org/10.1016/j.ssci.2014.11.018]
58. Pronovost, P.; Nolan, T.; Zeger, S.; Miller, M.; Rubin, H. How can clinicians measure safety and quality in acute care?. Lancet; 2004; 363, pp. 1061-1067. [DOI: https://dx.doi.org/10.1016/S0140-6736(04)15843-1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/15051287]
59. Martin, S.; Smith, P.C. Multiple public service performance indicators: Toward an integrated statistical approach. J. Public Adm. Res. Theory; 2005; 15, pp. 599-613. [DOI: https://dx.doi.org/10.1093/jopart/mui036]
60. Boulkedid, R.; Abdoul, H.; Loustau, M.; Sibony, O.; Alberti, C. Using and Reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review. PLoS ONE; 2011; 6, e20476. [DOI: https://dx.doi.org/10.1371/journal.pone.0020476]
61. Northcott, H.C.; Harvey, M.D. Public perceptions of key performance indicators of healthcare in Alberta, Canada. Int. J. Qual. Health Care; 2012; 24, pp. 214-223. [DOI: https://dx.doi.org/10.1093/intqhc/mzs012]
62. Mainz, J. Developing evidence-based clinical indicators: A state of the art methods primer. Int. J. Qual. Health Care; 2003; 15, pp. i5-i11. [DOI: https://dx.doi.org/10.1093/intqhc/mzg084]
63. Runciman, W.; Hibbert, P.; Thomson, R.; Van Der Schaaf, T.; Sherman, H.; Lewalle, P. Towards an International Classification for Patient Safety: Key concepts and terms. Int. J. Qual. Health Care; 2009; 21, pp. 18-26. [DOI: https://dx.doi.org/10.1093/intqhc/mzn057]
64. Tahir, A.C.; Darton, R. The Process Analysis Method of selecting indicators to quantify the sustainability performance of a business operation. J. Clean. Prod.; 2010; 18, pp. 1598-1607. [DOI: https://dx.doi.org/10.1016/j.jclepro.2010.07.012]
65. Scopetti, M.; Padovano, M.; Manetti, F.; Gatto, V.; D’errico, S.; Santurro, A.; Frati, P.; Fineschi, V. Key Performance Indicators in Claims Management: Definition of a Set of Indicators for the Evaluation of the Medico-Legal Activity. Curr. Pharm. Biotechnol.; 2021; 22, pp. 1971-1984. [DOI: https://dx.doi.org/10.2174/1389201022666210916124851]
66. Mant, J. Process versus outcome indicators in the assessment of quality of health care. Int. J. Qual. Health Care; 2001; 13, pp. 475-480. [DOI: https://dx.doi.org/10.1093/intqhc/13.6.475] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/11769750]
67. La Russa, R.; Viola, R.V.; D’errico, S.; Aromatario, M.; Maiese, A.; Anibaldi, P.; Napoli, C.; Frati, P.; Fineschi, V. Analysis of Inadequacies in Hospital Care through Medical Liability Litigation. Int. J. Environ. Res. Public Health; 2021; 18, 3425. [DOI: https://dx.doi.org/10.3390/ijerph18073425] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33806174]
68. Foti, F.; De-Giorgio, F.; Vetrugno, G. Let us learn from litigation claims: Actuarial tools can improve safety. BMJ; 2021; 373, n1480. [DOI: https://dx.doi.org/10.1136/bmj.n1480]
69. Santurro, A.; Vullo, A.M.; Borro, M.; Gentile, G.; LA Russa, R.; Simmaco, M.; Frati, P.; Fineschi, V. Personalized Medicine Applied to Forensic Sciences: New Advances and Perspectives for a Tailored Forensic Approach. Curr. Pharm. Biotechnol.; 2017; 18, pp. 263-273. [DOI: https://dx.doi.org/10.2174/1389201018666170207141525] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28176637]
70. Mandarelli, G.; Parmigiani, G.; Tarsitani, L.; Frati, P.; Biondi, M.; Ferracuti, S. The Relationship Between Executive Functions and Capacity to Consent to Treatment in Acute Psychiatric Hospitalization. J. Empir. Res. Hum. Res. Ethic; 2012; 7, pp. 63-70. [DOI: https://dx.doi.org/10.1525/jer.2012.7.5.63] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23324205]
71. Toccafondi, G.; Dagliana, G.; Fineschi, V.; Frati, P.; Tartaglia, R. Proactive Risk Assessment through FMEA of Home Parenteral Nutrition Care Processes: A Survey Analysis. Curr. Pharm. Biotechnol.; 2021; 22, pp. 433-441. [DOI: https://dx.doi.org/10.2174/1389201021666200612171943]
72. Vetrugno, G.; Foti, F.; Grassi, V.M.; De-Giorgio, F.; Cambieri, A.; Ghisellini, R.; Clemente, F.; Marchese, L.; Sabatelli, G.; Delogu, G. et al. Malpractice Claims and Incident Reporting: Two Faces of the Same Coin?. Int. J. Environ. Res. Public Health; 2022; 19, 16253. [DOI: https://dx.doi.org/10.3390/ijerph192316253]
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Delogu, Giuseppe 1 ; Raffaele La Russa 2
; Foti, Federica 3
; Grassi, Vincenzo M 3
; Vetrugno, Giuseppe 3 ; De Micco, Francesco 4
; De Benedictis, Anna 5
; Tambone, Vittoradolfo 4 ; Rinaldi, Raffaella 1 ; Frati, Paola 1
; Fineschi, Vittorio 1
1 Department of Anatomical, Histological, Forensic and Orthopaedic Sciences, Sapienza University of Rome, 00128 Rome, Italy
2 Department of Clinical and Experimental Medicine, University of Foggia, 71122 Foggia, Italy
3 Risk Management Unit, Fondazione Policlinico Universitario “A Gemelli” IRCCS—Legal Medicine, Department of Health Surveillance and Bioethics, Università Cattolica del Sacro Cuore, 00168 Rome, Italy
4 Research Unit of Bioethics and Humanities, Department of Medicine and Surgery, Università Campus 12 Bio-Medico di Roma, Via Alvaro del Portillo, 21, 00128 Roma, Italy; Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, Italy
5 Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo, 200, 00128 Roma, Italy; Research Unit of Nursing Science, Department of Medicine and Surgery, Università Campus Bio-Medico di Roma, Via Alvaro del Portillo, 21, 00128 Roma, Italy