Lilalva Scale: soft-hard technology to measure clinical competencies in emergencies of nurses

Headnote

ABSTRACT

Objectives: to present the visual identity of a soft-hard technology; to describe its technical characteristics, standardization and norms of criteria for the interpretation of results, in order to measure clinical competencies in emergencies of nurses. Methods: article developed as a methodological study for the creation of the graphic image of the scale and standardization based on previous studies of validity and reliability evidence. Results: the designed technology was named "Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses". The graphic image text is organized by title, instructions on how to fill it out, description of the quality of delivery, and behavioral actions with respective degrees of response. The Scale has 22 items with characterization data. The standardization of the measurement instrument included: instructions for use, calculation protocol, and hypothetical example to calculate scores, the sum of the resulting scores metric for each clinical competence and diagnostic classification. Conclusions: the visual identity and the standardization instrumentalize the use of the Scale for those interested in the theme.

Descriptors: Employee Performance Appraisal; Behavior Rating Scale; Clinical Competence; Emergency Nursing; Products Technology.

RESUMO

Objetivos: apresentar a identidade visual de uma tecnología leve-dura; descrever suas características técnicas, padronizaçâo na aplicaçâo e normatizaçâo de criterios para a interpretaçâo dos resultados, a fim de mensurar competencias clínicas em emergencias de enfermeiras e enfermeiros. Métodos: artigo desenvolvido como estudo metodológico de criaçâo da imagem gráfica da escala e estandardizaçâo calcada em estudos pregressos de evidencias de validade e confiabilidade. Resultados: a tecnologia desenhada foi denominada de "Escala Lilalva de Medida das Competencias Clínicas em Emergencias de Enfermeiras e Enfermeiros". O texto da imagem gráfica está organizado por título, instruçöes sobre a forma de preenchimento, descriçâo da qualidade da entrega e açöes comportamentais com respectivos graus de resposta. A Escala possui 22 itens com dados de caracterizaçâo. A estandardizaçâo do instrumento de medida contemplou: instruçöes para uso, protocolo de apuraçâo, exemplo hipotético para calcular escores, métrica da somatória dos escores resultantes para cada competencia clínica e classificaçâo diagnóstica. Conclusoes: a identidade visual e a estandardizaçâo instrumentalizam o uso da Escala aos interessados na temática.

Descritores: Avaliaçâo de Desempenho Profissional; Escala de Avaliaçâo Comportamental; Competencia Clínica; Enfermagem em Emergencia; Tecnologia de Produtos.

RESUMEN

Objetivos: presentar la identidad visual de una tecnología blanda-dura; describir sus características técnicas, estandarización en la aplicación y normalización de criterios para la interpretación de los resultados, a fin de mensurar competencias clínicas en urgencias de enfermeras y enfermeros. Métodos: artículo desarrollado como estudio metodológico de creación de la imagen gráfica de la escala y estandarización calcada en estudios anteriores de evidencias de validad y confiabilidad. Resultados: la tecnología diseñada fue denominada de "Escala Lilalva de Medida de las Competencias Clínicas en Urgencias de Enfermeras y Enfermeros". El texto de la imagen gráfica está organizado por título, instrucciones sobre la forma de relleno, descripción de la calidad de la entrega y acciones comportamentales con respectivos grados de respuesta. La Escala tiene 22 ítems con datos de caracterización. La estandarización del instrumento de medida contempló: instrucciones para uso, protocolo de averiguación, ejemplo hipotético para calcular escores, métrica de la sumatoria de los escores resultantes para cada competencia clínica y clasificación diagnóstica. Conclusiones: la identidad visual y la estandarización instrumentalizan el uso da Escala a los interesados en la temática.

Descriptores: Evaluación del Rendimiento de Empleados; Escala de Evaluación de la Conducta; Competencia Clínica; Enfermería de Urgencia; Tecnología de Productos.

How to cite this article:

INTRODUCTION

Scientific research to construct a tool to evaluate professional behavior involves two worlds: the world that the researcher theorizes and the observable world. In the health field, the needs of the client who goes to a clinic for a routine consultation are different from those who go there in an urgent or emergency situation. The person at real or potential risk, if there is no competent multi-professional team to assist him, may die within minutes. However, competently attending to someone at imminent risk of death is not an easy task. Among the various sociocultural factors involved, there are people management policies and practices that must be aligned, as a whole, with the organization's objectives, its values, and mission, being necessary to integrate organizational strategies and personal expectations to the management model(1).

Competencies are necessary for professionals to exercise this practice with excellence. However, identifying and measuring them is a great challenge, because, unlike knowledge and skills that can be measured by a written, oral, practical demonstration and/ or curricular assessment, the measurement of attitude/behavior is much more complex, because it is a subjective phenomenon.

Specifically in nursing, the search for the effectiveness of its people management processes by competencies is advancing more and more. Performance evaluation acquires an even more singular meaning, because it is also understood as an instrument of the work process for the improvement and adequacy of the care provided in health®. From this perspective, the evaluation focuses on the results of the person, the set of deliverables to the service and verifies, at a given degree of complexity and development axis, how much of what is expected has been delivered by the professional®.

It is necessary to have assessment instruments that point out deliveries of nurses and nurse workers, characterizing the complexity of their work and occupational environment. However, until the year 2011, there was no psychometric scale created or adapted in Brazil, published in journals, which measured competencies of nurses and nurses working in emergency services. In order to fill this gap, a soft-hard technology was built for this target population; and, due to its complexity and extent, some validation studies were gradually carried out to support the theoretical and empirical feasibility of this technology.

From the psychometric tradition, the first challenge in proposing a measurement instrument lies precisely in designing items for the legitimate, valid and accurate assessment of the constructs of interest. These instruments need to be recognized by the scientific community, based on the demonstration of evidence of varied validity for the use and interpretation of their scores; they must also be aligned with the underlying theoretical assumptions and with the contexts and practices intended for their use(4"5).

In psychometric studies, the literature recommends that, in addition to evidence of validity and accuracy, standardization and norms are also taken as basic parameters for a behavior measurement tool to be considered scientifically based. Standardization and norms define how the application of the instrument should be conducted, the audiences and contexts involved, how to determine the results, and how to interpret them(6_9). The technical standard, in this sense, is an essential factor for the interpretation of the results, although it is reported that less than a third of the instruments present all these conditions considered as minimum®4.

Given the above, it is important to formalize a scale that establishes common rules for all users, both for application and scoring00"®. It is also essential that people know how to use measurement instruments and interpret their results. In this context, it is possible to establish what and how appropriate is the use, purposes, types of information that are obtained, and how the results can be integrated into the actions and decisions of those involved. For this to occur, it is up to the researchers responsible for its construction to provide detailed instructions for the application and interpretation of the data(4). Therefore, the purpose of this article is to clarify doubts about the created scale, guide its use and interpretation of results in a correct and reliable way.

OBJECTIVES

To present the visual identity of a soft-hard technology, describe its technical characteristics, standardization and norms of criteria for the interpretation of results, in order to measure clinical competencies in emergencies of nurses.

METHODS

Ethical aspects

This research was submitted to and approved by the Institution's Ethics and Research Committee, followed national and international standards for ethics in research with human subjects, and is in compliance with Resolution 466/2012 of the National Health Council.

Type of study

This is an article developed as a methodological study of the construction of a light-hard technology, product of a thesis conducted at the Paulista School of Nursing of the Federal University of Sâo Paulo, completed in 2016. Specifically for this phase of the research, the results of the psychometric parameters of the evidence of validity and reliability of the Competence Scale of Actions of Nurses in Emergencies (CSANE) were used(12).

The Ninth Resolution of the Federal Council of Psychology was also used(10) as a reference for characterizing the Scale, standardizing its application, standardizing the criteria for interpreting the results obtained from the scores and degrees of clinical competence and hypothetical case.

Procedures

Based on the psychometric data evidenced in the CSANE, a visual identity was created, presented in a graphic version entitled "Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses". For this construction, some steps were followed. Initially, the content to be included in the results of the empirical analysis was established04. The data was organized by title, instructions on how to fill it out, description of the quality of delivery, behavioral actions with respective degrees of response, and metrics of the sum of the resulting scores for each clinical and overall competence. From this dataset drawn in Microsoft Office Word, a graphic image was created using Photoshop, Illustrator, and InDesign software - all Adobe Package 2020 application products. It is recommended that the artwork be printed on two sheets of A4 white sulfite paper.

The technical characteristics of the Lilalva Scale were presented in a summary sheet, which allows the reader to assimilate and understand the essence between the theorized and empirically evidenced. Also established were norms of use considered common to all users in relation to the uniformity of procedures and interpretation of data.

RESULTS

The results of this soft-hard technology created were described in five topics: technical characteristics and description of the Lilalva Scale; instructions for use of the Scale; measurement protocol; diagnostic classification and hypothetical example; and finally the norms for diagnostic classification of the Degree of Clinical Competence in Emergencies.

Presentation of the Clinical Competence Measurement Scale in Emergencies

The scale created and validated, entitled "Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses", is a soft-hard technology that allows measuring the presence and degree of Clinical Competence. "Clinical Competence" in emergencies is understood as the nurses' ability to provide humanized care to a person in an emergency situation with professional excellence at the right time and in the right place.

The purpose of the scale is the diagnostic classification of the quality of care delivery, according to the rater's perception. It originally presents a version that can be self-applied or applied by another person in both self-assessment and hetero-assessment.

It can be used in two scenarios: in the professional environment, to evaluate nurses who work in direct care to people in emergency services; and in the academic environment, for the development of scientific research. In both contexts of the Scale's use, the fundamental ethical precepts of autonomy, beneficence, non-maleficence, justice and equity must be followed031.

Technical characteristics and description of the Lilalva Scale

The Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses is composed of seven Clinical Competency dimensions, eight Basic Competencies, 32 Associated Competencies, and 78 behavioral items/actions. The seven latent Clinical Competency dimensions were named Professional Practice, Relationships at Work, Positive Challenge, Directing Action, Constructive Conduct, Professional Excellence, and Adaptation to Change. Each Clinical Competency is underpinned by Basic Competencies, Associated Competencies, and behavioral items/actions.

Given the fact that there are 78 items/actions, we chose to randomize them, in order to reduce response bias related to the indiscriminate selection of the respondent, which could influence the diagnosis. To this end, a table of random numbers was used. The Arabic number preceding each item corresponds to its position in the scale, according to the draw.

The Scale has 22 items with data that characterize it, according to the synthesis sheet (Chart 1).

The Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses makes it possible to register representative actions of Clinical Competencies, as presented in Figure 1.

Considering that the scale has the items and degrees, but the seven clinical dimensions are not explicit in the printed tool, as well as their respective Basic and Associated Competencies, it is necessary to consult the number of the corresponding item/action to calculate the scores and classify the degree of competency (Chart 2).

The reader interested in details about the psychometric properties obtained in the scale's validation research may consult the study available online(12).

Instructions for using the scale

Application

Regarding application, the scale should be printed on sulfite paper and answered with a ballpoint pen. The diagnostic assessment of clinical competence should be performed in a private and quiet environment. The content of the scale is organized by title, instructions on how to fill it out, description of the quality of delivery, behavioral actions with respective degrees of response, and sum of the resulting scores for each clinical competence. Filling out is very simple and should be done on printed paper. The respondent need only mark with the letter X the degree of competence he/she considers to have or that the other has for the described action/item.

Calculation of response scores for behavioral item/action

The behavioral actions of nurses and nurse practitioners can be measured using a five-point Likert-type response scale. The score ranges from 1 to 5 points (from "Not at all competent" to "Extremely competent")041. The degree of Competence represents the quality of care delivery, as follows041:

Degree 1/Not Competent at all: Rarely performs the described action in part of his daily practice, having knowledge deficit and not using appropriate techniques, with the need for constant supervision in the low complexity actions, so that he can achieve the goals recommended in the plans(14).

Degree 2/Little Competent: Sometimes he does the described action in his daily practice with a certain deficit of knowledge and, frequently, without the use of appropriate techniques, in a way that is not always independent, therefore with the need for supervision in the care and management actions of medium complexity, in order to achieve most of the objectives recommended to execute it(14).

Degree 3/Competent: Often does the described action in his daily practice with sufficient knowledge and use of appropriate techniques, independently and with the need for supervision exclusively in the managerial actions to achieve the objectives set to execute it(14).

Degree 4/Very Competent: Almost always does the described action in his daily practice with the desired knowledge and use of appropriate techniques, independently, but with eventual need for supervision only in the more complex managerial actions, in order to achieve the objectives set for executing it(14).

Degree 5/Extremely Competent: Always does the described action in his daily practice with all the necessary knowledge and use of appropriate techniques, independently and without any need for supervision when doing the managerial and assisting activities in all degrees of complexity, reaching the goals recommended to execute it(14).

Calculating the Clinical Competence Score by latent dimension

It is a measure of the degree of competence for each of the seven dimensions. The calculation is made based on the value marked for each item, by means of a simple sum. Since the measurement must be made for each dimension individually, it is necessary to identify (Figure 2) the Arabic number corresponding to the item/action and the respective value of the response degree. With the values assigned by the respondent on the response scale, a mathematical operation of addition is performed. The result represents the score of competence of nurses for the respective dimension.

Calculation protocol, diagnostic classification, and hypothetical example of how to calculate scores and make the diagnostic classification

The Clinical Competence Matrix/Core is a framework of quality of care delivery by nurses working in urgency and emergency services. The data make it possible to make a specific diagnostic classification of the professional's latent trait by dimension and item by item, according to the hypothetical parameters presented in Figure 2.

An example of how to proceed with the calculation of the scores of a clinical competence for Professional Excellence and for the General Competence Score is presented, based on the hypothetical case illustrated in Figure 2. These are examples of how to calculate the scores of the dimensions, which should be replicated in the other dimensions of the scale, as well as in the General Competence Score.

Calculating Clinical Competence in Professional Excellence

This latent trait has four actions/items: 1 - Constantly updates knowledge in emergencies; 2 - Performs nursing diagnoses; 3 - Participates in realistic simulation; 4 - Uses other opportunities for professional development. These items, according to Figure 1, are respectively 04, 29, 31 and 55, and the values of the hypothetical response degrees were: Item 04 = 5; Item 29 = 1; Item 31 = 5; Item 55 = 4. The step-by-step calculation is shown below:

* Step 1: Identification of items 04, 29, 31 and 55;

* Step 2: Checking the values of answer degrees 5, 1,5 and 4;

* Step 3: Add the addition symbol (+) between the degrees - 5+1+5+4;

* Step 4: Sum all degrees - 5+1+5+4 = 15.

The value 15 is the score resulting from the sum of the four items and represents the degree of competence for the latent trait Professional Excellence.

Calculating the Overall Score of Clinical Competence in Emergencies

It is the measure of the sum of the scores of all the latent dimensions of the Lilalva Scale. The calculation of the degree of competence is made based on the values of the scores resulting from the mathematical operations of addition of each of the seven dimensions. Since there are seven dimensions, it is necessary to have the values of the sum of the scores of each one. As a hypothetical example: Professional Practice = 146; Relationships at Work = 55; Positive Challenge = 46; Targeted Action = 26; Constructive Attitude = 10; Professional Excellence = 15; and Adaptation to Change = 12. With these values, it is possible to do the mathematical operation of addition. Thus, Clinical Competence is the product of the sum of the scores of all the latent dimensions of a scale:

Clinical Competence Score = 146 + 55 + 46 + 26 + 10 + 15 + 12 = 310.

This value of 310, the result of the sum of the scores of the seven dimensions, represents the degree of Clinical Competence of the nurses to act in emergency services. Once the seven scores that represent the clinical competencies and the General Competency Score have been calculated, the next step is to proceed to the diagnostic classification of the degree of competency, as detailed below.

Standards for Diagnostic Degree of Clinical Competence Classification in Emergencies

Degree of Competence is done based on the result of the score calculation. For each dimension, there is a minimum and a maximum score (Figure 3), which was calculated according to the total number of items and the five possible degrees of response.

An example of this is that in the dimension Relationships at Work, the score 55 represents those nurses are competent. The latent traits in Targeted Action, Professional Excellence, and Adaptation to Change, whose respective scores were 26, 15, and 12, are classified as very competent. The Clinical Competencies Professional Practice, Positive Challenge and Constructive Attitude, with respective scores of 146, 46 and 10, show that the degree of quality of delivery is extremely competent.

From the point of view of the global diagnosis of our hypothetical case, in which the nurses obtained a total score equal to 310, it is possible to affirm that this person has the degree of Clinical Competence classified as very competent; and, according to the dimensions, the degree varies from competent to extremely competent.

Considering the hypothetical example, by visualizing the matrix/core structure design (Figure 2), it is possible to identify, reflect, and better understand the strengths and weaknesses of the person being evaluated. From this framework, attention is drawn to items 20, 25, 29, 34, and 46 assigned with Degree 1, highlighted in red; and items 17, 57, and 62 corresponding to Degree 2, in yellow. Such actions show degrees of deficiencies, whose competencies involve socio-emotional values, as presented in Figure 2. Given the classificatory nature, this data can benefit nurses, managers, and the services, indicating adjustments, training needs, and work conditions experienced®.

DISCUSSION

In this study, the Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses was presented, as well as the standardization, the norm and a hypothetical example for its use.

The construction of valid and reliable scales has been advocated because it allows for the systematic collection of data and quantitative assessment of phenomena, enabling the correlation of variables through statistical tests. Certainly, the conceptual and psychometric attributes are necessary and important to ensure objective and reliable measurement results051. However, following the example of what has been built in psychology, it is also extremely important to consider standardization and norms of the use of scales so that they can be well used. The lack of understanding of how to use them, as well as filling them out incorrectly, makes the data unreliable enough to analyze and interpret them(16).

It is believed that with a detailed description and hypothetical example, it is possible to make behavioral diagnoses that are closer to the true latent trait, both in self-assessment and in hetero-assessment. As exemplified in the hypothetical case, the person was diagnosed with Degree 4 of General Clinical Competence, that is, very competent. However, when analyzing the Matrix/Core design, even if the person is extremely competent to provide patient care in an emergency situation, one can see weaknesses, especially in the Clinical Competence Relationships at Work, in the Basic Competence Leadership, and in the Associated Competence Emotional Balance - competences that require careful attention to achieve professional excellence. It is possible to observe behaviors and measure differences between individuals or between reactions of the same individual on different occasions in a standardized way(10-11).

Considering that the process of demonstrating evidence of validity and accuracy for measuring instruments is procedural and continuous, it is suggested that new studies be conducted with the Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses in different contexts where nurses work. It is hoped that the results of these studies will be published so that the scientific and professional community can follow the advances of research on this theme, as well as encourage both discussion on the subject and the professional development of nurses involved in the process.

Study limitations

Although there are strengths, the Lilava Scale has limitations in relation to the time it takes to complete the questionnaire because of the number of items. In addition, the artwork was created to be printed on sheets of sulfite paper, but a new study is being conducted by the creators to computerize it.

Contributions to the field of Nursing

Having the standardization and norms of use of a scale exemplified with hypothetical data and freely available to those interested in the subject is a great advance in management by nursing competence. Also, to allow its applicability by local managers in continuing education and in the elaboration of performance indicators is of unparalleled relevance for the evaluation of frontline professionals. In other areas, such as psychology, most measurement instruments like the one presented here are marketed by large publishers specialized in psychological tests. Since the Lilalva Scale allows self-assessment, it is believed to be a valuable tool for the person him/herself to monitor his/her performance, seeking strategies to reduce weaknesses. In addition, the comprehensiveness of the Scale validation sample and the robustness of the evidence aggregate, besides the sociocultural variations, the differences in the professional profile of the nurses who work in emergency services in the five regions of Brazil(12).

CONCLUSIONS

The Lilalva Scale for Measuring Clinical Competencies in Emergencies of Nurses is a soft-hard technology for measuring competencies, with evidence of validity and reliability, defined standards for its application, and well-detailed rules on how to use the results obtained from its application for the diagnostic classification of the clinical competencies of nurses. This is a psychometric instrument of robust quality, anchored in a current and contextualized minitheory and made freely available to researchers interested in the topic of competencies and professional development in nursing. SUPPLEMENTARY MATERIAL

https://doi.org/10.1590/1982-0194201400062

FUNDING

This research was funded by the National Council for Scientific and Technological Development (CNPq), process no. 483.449/2013-0.

ACKNOWLEDGMENT

Acknowledgement for Designer Jullia Tavares Valério and Web Designer Leandro Santos de Oliveira.

Sidebar

Holanda FL, Marra CC, Meireles ECA, Balsanelli AP, Cunha ICKO. Lilalva Scale: soft-hard technology to measure clinical competencies in emergencies of nurses. Rev Bras Enferm. 2022;75(5):e20210950. https://doi.org/10.1590/0034-7167-2021-0950

Corresponding author:

Flávia Lilalva de Holanda

E-mail: [email protected]

EDITOR IN CHIEF: Antonio José de Almeida Filho

ASSOCIATE EDITOR: Priscilla Valladares Broca

Submission: 02-19-2022 Approval: 03-31-2022

References

REFERENCES

1. Ministério da Saúde (BR). Portaría n° 1.600, de 7 de julho de 2011. Reformula a Política Nacional de Atençâo as Urgencias e institui a Rede de Atençâo as Urgencias no Sistema Único de Saúde (SUS) [Internet]. Brasilia: Ministério da Saúde; 2011 [cited 2021 Nov 11]. Available from: http://bvsms.saude.gov.br/bvs/saudelegis/gm/2011/prt1600_07_07_2011.html

2. Pierantoni CR, França T, Ney MS, Monteiro VO, Varella TC, Santos MR, et al. Avaliaçâo de desempenho: discutindo a tecnología para o planejamento e gestâo de recursos humanos em saúde. Rev Esc Enferm USP;45(esp):1627-31. https://doi.org/10.1590/ S0080-62342011000700014

3. Dutra JS. Competencias: conceitos e instrumentos para a gestâo de pessoas na empresa moderna. Sâo Paulo: Atlas; 2013.

4. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington: American Educational Research Association; 2014.

5. Markus KA, Borsboom D. Frontiers of Test Validity Theory: measurement, causation and meaning. New York: Routledge; 2013.

6. Noronha APP, Primi R, Alchieri JC. Parámetros psicométricos: uma análise de testes psicológicos comercializados no Brasil. Psicol Cienc Prof. 2004;24(4):88-9. https://doi.org/10.1590/S1414-98932004000400011

7. Cardoso LM, Silva-Filho JH. Satepsi e a qualidade técnica dos testes psicológicos no Brasil. Psicol Cienc Prof. 2018;38(n.esp.):40-9. https://doi. org/10.1590/1982-3703000209112

8. Primi R, Nunes CHS. Aspectos técnicos e conceituais da ficha de avaliaçâo dos testes psicológicos. In: Conselho Federal de Psicologia. Avaliaçâo psicológica: diretrizes na regulamentaçâo da profissâo [Internet]. Brasilia; 2010[cited 2021 Nov 11]. p.101-21. Available from: https://satepsi.cfp.org.br/docs/Diretrizes.pdf

9. Nunes CHS, Primi RO. Satepsi: desafios e propostas de aprimoramento. In: Conselho Federal de Psicologia. Avaliaçâo psicológica: diretrizes na regulamentaçâo da profissâo [Internet]. Brasilia; 2010[cited 2021 Nov 11]. p.129-48. Available from: https://satepsi.cfp.org.br/docs/ Diretrizes.pdf

10. Conselho Federal de Psicologia. Resoluçâo N° 9, de 25 de abril de 2018. Diretrizes para a realizaçâo de Avaliaçâo Psicológica no exercício professional da psicóloga e do psicólogo, regulamenta o Sistema de Avaliaçâo de Testes Psicológicos - SATEPSI [Internet]. 2018[cited 2021 Nov 11]. Available from: http://crp11.org.br/upload/Resolu%C3%A7%C3%A3o-CFP-n%C2%BA-09-2018-com-anexo.pdf

11. Andrade JM, Valentini F. Diretrizes para a construçâo de testes psicológicos: a Resoluçâo CFP n° 009/2018 em destaque. Psicol Cienc Prof. 2018;38(n.esp):28-39. https://doi.org/10.1590/1982-3703000208890

12. Holanda FL, Marra CC, Cunha ICKO. Evidence of validity of the Competence Scale of Actions of Nurses in Emergencies. Rev Latino-Am Enfermagem. 2019;27:e3128. https://doi.org/10.1590/1518-8345.2814.3128

13. Beauchamp T, Childress J. Principles of Biomedical Ethics. 4a ed. New York: Oxford University Press; 1994.

14. Holanda FL, Marra CC, Cunha ICKO. Assessment of professional competence of nurses in emergencies: created and validated instrument. Rev Bras Enferm. 2018;71(4):1865-74. https://doi.org/10.1590/0034-7167-2016-0595

15. Vendramini CMM, Lopes FL. Leitura de manuais de testes psicológicos por estudantes e profissionais de psicologia. Aval Psicol [Internet]. 2008[cited 2021 Nov 11];7(1):93-105. Available from: http://pepsic.bvsalud.org/pdf/avp/v7n1/v7n1a12.pdf

16. Feitosa MC, Soares LS, Beleza CMF, Silva GRF, Leite IRL. Uso de escalas/testes como instrumentos de coleta de dados em pesquisas quantitativas em enfermagem. SANARE[Internet]. 2014 [cited 2021 Nov 11];13(2):92-7. Available from: https://sanare.emnuvens.com.br/ sanare/article/view/579/312