Construction and validation of a distance Basic Life Support Course

Abstract

Initially, we sought to identify learning needs, definition of instructional objectives, target audience, resources available for course development. [...]in response to an emerging need of the professionals of the Hospital Universitario Onofre Lopes (HUOL), in the northeast of Brazil, the BLSC setting was defined as a priority at this time for all professional categories and academics attending the hospital. [...]we have as a psychomotor objective the navigation in the LMS, the technical abilities of execution of cardiopulmonary resuscitation maneuvers and disengagement maneuvers taught by video, besides the simulated game at the end of the course (educational game). For Ausubel(6), learning is a process in which an individual relates (anchor) new information in a substantive (non-literal) and non-arbitrary way with relevant aspects of their cognitive structure·6'13·, forming, from of this interaction, a "kind of conceptual hierarchy", in which more specific elements of knowledge are linked to more general and inclusive concepts, ideas or propositions·13·. [...]for the meaningful learning to occur, it is fundamental to have three conditions. 1st- previous knowledge - called by Ausubel, as subsumption, which can be an image, symbol, concept or proposition - already existing in the learner's cognitive structure, which enable them to connect to new information. 2nd- explicit attitude of student learning. [...]the BLSC was thought based on the needs of the health professionals identified by the Hospital Management, thus making it possible to develop the latter in a subject of great interest of this public, obeying condition number 2 of the MĽT·6·.

Headnote

Submission: 03-04-2018 Approval: 05-25-2018

ABSTRACT

Objective: to describe the content construction and validation process of the Distance Education Basic Life Support Course. Method: methodological study, developed through literature review, outlined in the light of the Bloom's Taxonomy and Ausubel's Meaningful Learning Theory. For validation, the analysis was performed with judges, using a structured tool. Results: the construction of the distance course was complex and challenging, since it was tried to develop it with logical-methodological coherence, considering the constructivist perspective, representing an advance in the process of dissemination of the Urgency and Emergency teaching culture. As for the content validation process, it was verified that of the 16 suggestions made by the judges, 14 were accepted and two rejected. Conclusion: the course had its contents validated by experts.

Descriptors: Cardiopulmonary Resuscitation; Validation Studies; Distance Education; Teaching; Educational Technology; Emergency Nursing.

RESUMO

Objetivo: descrever o processo de construção e validação de conteúdo do curso de Suporte Básico de Vida na modalidade Educação a Distancia. Método: estudo metodológico, desenvolvido mediante revisâo da literatura, delineado a luz da Taxonomia de Bloom e Teoria da Aprendizagem Significativa de Ausubel. Para validação, foi realizada a análise com juízes, mediante utilização de instrumento estruturado. Resultados: a construção do curso a distancia se mostrou complexa e desafiadora, uma vez que se procurou desenvolve-lo com coerencia lógico-metodológica, considerando a perspectiva construtivista, representando um avanço no processo de disseminação da cultura do ensino de Urgencia e Emergencia. Quanto ao processo de validação de conteúdo, verificou-se que das 16 sugestöes realizadas pelos juízes, 14 foram acatadas e duas rejeitadas. Conclusâo: o curso obteve o seu conteúdo validado por especialistas.

Descritores: Ressuscitação Cardiopulmonar; Estudos de Validação; Educação a Distancia; Tecnología Educacional; Enfermagem em Emergencia.

RESUMEN

Objetivo: describir el proceso de construcción y validación de contenido de un curso de Soporte Básico a la Vida en la modalidad de educación a distancia. Método: estudio metodológico, desarrollado mediante revisión de la literatura, delineado a la luz de la Taxonomía de Bloom y la Teoría del Aprendizaje Signifi cativo de Ausubel. Para validación, fue realizado un análisis con jueces mediante la utilización de instrumento estructurado. Resultados: la construcción de un curso a distancia se mostró compleja y desafi ante una vez que se intentó desarrollarlo con coherencia lógico-metodológica, considerando la perspectiva constructivista, representando un avance en el proceso de diseminación de la cultura de la enseñanza de urgencia y emergencia. En cuanto al proceso de validación de contenido, se verificó que de las 16 sugerencias realizadas por los jueces, 14 fueron acatadas y 2, rechazadas. Conclusión: el curso obtuvo su contenido validado por expertos.

Descriptores: Reanimación Cardiopulmonar; Estudios de Validación; Educación a Distancia; Tecnología Educacional; Enfermería de Urgencia.

INTRODUCTION

Constant changes with emphasis on patient safety show the need for training professionals to assure the quality of care provided to the population. Urgency care, including Basic Life Support (BLS), is an inherent aspect of the health professional's practice and demands regular and timely training, with specific knowledge, technical skills and relational skills required for the effective attendance of a Cardiorespiratory Arrest (CRA)(1).

The BLS needs disclosure as it is a protocol for easy learning and execution that can be performed not only by a health professional, but also by any properly trained person. The greater the number of people prepared to assist CRA victims, there will be improvement in statistics and a significant reduction in morbidity and mortality rates(1).

Faced with the pace of modern life, and possibilities of access and dissemination of information, the Distance Education has been gaining prominence worldwide as an innovative method in the programs of Permanent Education. Distance Education courses take place through information and communication technologies' use, having as main advantages the interactivity, flexibility and possibility of the student to manage their place and study times according to their needs(2). This is even more important when it comes to health professionals who have temporary difficulties, given many employments they often end up undertaking.

However, the development of distance learning courses is not an easy task. This new pedagogical domain, which integrates multiple media and teaching aids, requires a set of steps that encompass processes that must be planned and integrated in order to enable meaningful learning(3).

The final stage of the Distance Education course development is assessment, where experts, through predefined criteria, will validate the Learning Management System (LMS) as a whole, test its functioning and assess its pros and cons in a way that it can achieve the expected results(2).

However, turning to the assessment of quality of teaching and learning in Distance Education can encompass different elements that structure and compose the design of a course in that mode. It is believed that studies of this nature provide the opportunity to acquire the necessary knowledge to standardize language and care for the CRA victim, reducing the loss of time and offering a greater perspective of survival and fewer sequels.

OBJECTIVE

To describe the content construction and validation process of a distance BLS course.

METHOD

Ethical Aspects

Ethical precepts established by Resolution 466/2012 of the Conselho Nacional de Saúde (National Health Council) were followed, so that the proposal of the study was assessed and approved by the Ethics and Research Committee of the Hospital Universitario Onofre Lopes (HUOL). In addition, it is emphasized that the participation of the patients in the research was conditioned by the signing of the Informed Consent Term.

Design, place of study and period

This is a methodological study, with a quantitative approach and data analysis, to describe the content construction and validation process of a distance Basic Life Support Course (BLSC) to support the learning of academics and health professionals.

The content validation process of the BLSC was delineated from the theoretical pole of the Pasquali model, in two stages(4). The first stage was construction, and the second was validation. The construction was carried out in August 2016, by a team composed of three-selected experts, intentionally, being determined as criterion of inclusion of extensive experience in the Basic Life Support theme, with research and studies published in the area.

Content assessment occurred in September 2016, through a tool with six expert judges in the areas of Nursing and Medicine, according to the recommendation of a theoretical framework in the area of Methodological Studies(4). As a criterion for the selection of specialists, an adaptation of Fehring's scoring system was developed for this study(5), with a minimum score of five points for experts selection.

Population or sampling; inclusion and exclusion criteria

Thus, it was considered a criterion of inclusion of the judges: to be master in Nursing or Health Sciences (2 points); to be instructor/professor of Basic Life Support in University or in Permanent Education Centers (2 points); to have research in the area of Teaching-Learning Methods or Basic Life Support (1 point); to have article published in the area of Teaching-Learning Methods or Basic Life Support in a reference periodical (2 points); to have doctorate with thesis in the area of Teaching-Learning Methodologies or Basic Life Support (2 points); to have clinical practice experience of at least 05 years in hospital or pre-hospital emergency (3 points); to have certificate of specialization or residency in the area of Emergency/Intensive/Pre-Hospital Therapy (2 points). As an exclusion criterion, we have: to be an expert who has modified his/her research line 5 years ago and does not work on the theme or is away from professional practice/education for more than two years.

The tool had variables related to personal and professional characteristics and then the judge assessed 13 questions related to the course content: 1 - overall content assessment, 2 - organization, 3 - presentation, 4 - usefulness/pertinence, 5 - consistency , 6 - clarity, 7 - objectivity, 8 - reliability, 9 - feasibility, 10 - update, 11 - vocabulary, 12 - instructional sequence of topics; and 13 - learning assessment^.

Study protocol

For the development of BLSC, a literature review and the most updated international American Heart Association (AHA) protocols of BLSC were published in 2015 and as a pedagogical theory of contribution to educational conceptions, the BLSC was delineated in the light of the Bloom's Taxonomy(6), and Ausubel's Meaningful Learning(7), for content's elaboration.

The toll assessment occurred from the classification of each item in "adequate", "adequate with changes" and "inadequate". In the last two cases, motives or problems with items should be explicit, and there should be suggestions, so that they could be improved. The judges were then asked to rate each unit on a Likert scale of 1 to 10, with 1 being the lowest possible quality and 10 being the highest possible quality. At the end, the judge shall do an overall analysis of the course content, indicating his/her opinion to use the contents of each module.

After the data collection phase, the information obtained through the search form was organized into a Microsoft Office for Windows Excel spreadsheet, and was subsequently inserted into a SPSS software database, version 20.0, for Windows.

Results analysis and statistics

We used Cohen's Kappa Coefficient (K) to analyze the data, and measure the level of concordance and consistency of the judges in relation to whether or not the content remained. This was calculated using free and online software, the Online Kappa Calculator®, taking into account the "adequate with changes" indications for them. As acceptance criterion, concordance was higher than 0.65 among the judges, being considered good(9_10).

We also used the Content Validity Index (CVI), which measures the judges' agreement on the representativeness of the items in relation to the content under study. As acceptable, we considered indexes higher than 0.80 both for each unit assessment and overall course assessment(9-10).

RESULTS

First stage was construction. Initially, we sought to identify learning needs, definition of instructional objectives, target audience, resources available for course development. Thus, in response to an emerging need of the professionals of the Hospital Universitario Onofre Lopes (HUOL), in the northeast of Brazil, the BLSC setting was defined as a priority at this time for all professional categories and academics attending the hospital.

The Ministry of Health, the Health Technology Innovation Lab (LAIS, in Brazil) of the Universidade Federal do Rio Grande do Norte (UFRN), the Office of Distance Education (SEDIS, in Brazil) of UFRN, the Mobile Emergency Service of the Rio Grande do Norte State and the researcher herself provided material and financial resources required for the construction of the course and its allocation in the Learning Management System of the Sistema Único de Saúde [Unified Health System] (AVASUS) platform.

In this stage, the expert training was still carried out according to the structure and organization of a Distance Education course for AVASUS platform, and the didactic resources(11) were presented, which could be used in the course modeling.

For this, some theoretical frameworks were used in the course development, such as Bloom's Taxonomy® and Ausubel's Meaningful Learning Theory®.

We used the Bloom's Taxonomy to elaborate educational objectives, in order to guide the development of the material and assist the teaching-learning process assessment.

Cognitive educational objectives were defined as: defining the main concepts in BLSC; identifying different methods of Cardiorespiratory Resuscitation over the years; comparing the main changes in the guidelines in cardiorespiratory resuscitation; describing some aspects of the anatomy and functioning of the heart; defining Cardiorespiratory Arrest (CRA); identifying clinical signs of an CRA; identifying the main rhythms of CRA; distinguishing shocking rhythms from non-shocking rhythms; defining what is an Automated External Defibrillator (AED); describing the care in the use of AED; and identifying signs of partial and total obstruction in adults, children and infants.

The affective educational goals were: discerning when to suspend resuscitation efforts; executing step by step in the use of AED; operating the AED in special situations; coordinating actions between CPR and AED. All these goals seek to encourage problem-solving and critical thinking with significant emotional control.

Finally, we have as a psychomotor objective the navigation in the LMS, the technical abilities of execution of cardiopulmonary resuscitation maneuvers and disengagement maneuvers taught by video, besides the simulated game at the end of the course (educational game).

Summing up all the proposed objectives, the BLSC aims to present Basic Life Support actions recommended by the American Heart Association(12), in the care of Cardiorespiratory Arrest, Respiratory Arrest (RA) and Foreign Body Airway Obstruction (FBAO) victims. BLSC also fosters interactivity, autonomy, development of technical and behavioral competences, and support for access to educational technologies in the proposed setting (didactic material, case studies, self-instructional questions with automated correction with feedback to the participants of the course). Upon completion of the course, depending on the work/teaching institution of each participant, the student will be able to develop the skills taught in the videos of this course in practical stations and simulated scenarios.

Regarding the content of BLSC units, the recently published American Heart Association protocols were used, based on studies of strong scientific evidence, as well as collections of articles and chapters of books, clinical evidence results from international studies(12).

The elaboration of the educational material provided priority to the gradual complexity of knowledge in the modular contents, with meticulous care in the organization of the texts to make learning easier to the student. In addition, the whole content was structured with different didactic and technological strategies in an attempt to favor multisensory learning experiences. Chart 1 synthetizes the themes studied in each unit, overall objective, items/contents and didactic resources used.

The second step corresponded to the validation of the entire content of the BLSC. Of the six judges who assessed the tool, 50% are female, with a mean age of 41.3 (± 6.47) years, nurses (66.7%), work in the Emergency Medical Services (83, 3%) or in other pre-hospital services, they have masters degrees as a higher degree (50%). In terms of professional experience, judges have an average of 14.5 (± 6.35) years of care experience, 10 (± 3.43) years of Urgency and 10 (± 8.07) years of teaching.

In this process, 13 content assessment requirements adapted from Pasquali were taken into account, which were later analyzed through the CVI (Table 1).

In Chart 2, the presented requirements of each unit assessed as "adequate with changes", and gave suggestions for improvement or reformulation of the items. It was verified that, of the 16 suggestions made by the judges to improve the units, 14 were accepted and only 2 rejected. Most suggestions were related to the overall content assessment requirements, usefulness, clarity, objectivity and vocabulary. It should be noted that no item was found to be inappropriate by judges.

Kappa was also used and analyzed by unit, in which all indexs were considered between "good" and "perfect", with Unit 1 presenting a lower value (0.74), followed by Unit 3 (0.82) , Units 2 and 4 with (0.87 each). The BLSC content in its totality presented Kappa of 0.87.

However, when analyzing the units qualitatively on a scale of 1 to 10, Unit 2 presented the lowest mean, with 9.67 (± 0.52), Unit 4 with 9.70 (± 0.52), Unit 3 with 9.80 (± 0.41) and Unit 1 with a mean score of 10.00. The BLSC content presented a final average of 9.50 by the judges.

DISCUSSION

In the context of development and validation of technological educational courses, it is valid to analyze the theoretical contribution to the incorporation of these strategies in the teaching environments, which must be understood and consolidated from its conception.

In this sense, the preparation of this course counted on the theoretical contribution of Ausubel and Bloom, in addition to the most up-to-date scientific evidence in BLSC. For Ausubel(6), learning is a process in which an individual relates (anchor) new information in a substantive (non-literal) and non-arbitrary way with relevant aspects of their cognitive structure·6'13·, forming, from of this interaction, a "kind of conceptual hierarchy", in which more specific elements of knowledge are linked to more general and inclusive concepts, ideas or propositions·13·.

Thus, for the meaningful learning to occur, it is fundamental to have three conditions. 1st- previous knowledge - called by Ausubel, as subsumption, which can be an image, symbol, concept or proposition - already existing in the learner's cognitive structure, which enable them to connect to new information. 2nd- explicit attitude of student learning. And 3rd- the supply of a new knowledge structured in a logical, sequential and hierarchical way(14).

Regarding the first condition of Meaningful Learning Theory (MLT), it can be said that any person structured the entire content course from problem situations normally experienced by any health professional, since they always address real situations that happen every day in the life of the human being.

An example of this is the problem situation (initial video of the course) that addresses the life of Mr. Ernani, 42 years old, and like most Brazilians, has a very busy work routine and consumes a lot of his time, not having time to do any kind of regular physical activity. From this case, the course is structured, which is resumed in all modules in a logical and sequential way, showing the evolution to CRA and its treatment in the BLSC. This strategy allows the resumption of the student's cognitive structure of concepts, making possible the connection to the new information·13·.

Thus, the BLSC was thought based on the needs of the health professionals identified by the Hospital Management, thus making it possible to develop the latter in a subject of great interest of this public, obeying condition number 2 of the MĽT·6·.

It is important to confirm that the student's interest in learning to learn is one of the three conditions for MĽT·6'13·. It is necessary, besides elaborating a course that is in accordance with the needs of the target public, to be attentive, during the organization of the instructional material, to the (didactic and technological) mechanisms that may allow the awakening of that interest. Thus, the didactic strategies addressed in addition to increasing student interest, promote the development of new subsumption and/or strengthening of existing ones(14).

The content proposed in the BLSC was presented following a systematic, sequential and hierarchical logic of the main concepts, allowing the student to access the subsequent subject only when the previous unit was concluded, enhancing the participant's cognitive structure and maximizing his/her learning process, as Ausubel stresses in his/her 3rd MLT condition®. Still, according to the author, identifying the basic concepts, as well as their structuring, is the first and usually difficult task in the organization of teaching, based on the MLT.

Consideration should be given to the opportunity to implement BLSC in AVASUS. AVASUS is a virtual learning space, with free Distance Education courses, designed to qualify students and health professionals for training, management and assistance in the Unified Health System (SUS- Sistema Único de Saúde)(15). The platform allows the use of different media that can be accessed at any time, including the mobile version. The student has the freedom to build his/her own formative itinerary, selecting courses related to his/her area of training(16). Thus, the structure of AVASUS and all available technological resources feed conditions 2 and 3 of the MLT.

Regarding the validation stage, it is known that assessment is a substantial element of the teaching-learning process of a Distance Education project. It is the one that certifies its seriousness and establishes credibility, plays a very important role as a systematic tool of corrections of failures and promotion of correctness, being necessary so as not to lose learning guidelines, teaching objectives and the sustainability of the project(17).

As for the assessment processes, these were present in all units (formative assessment) and at the end of the course (summative assessment), using problem situations and clinical cases as valid and practical methods to find evidence of meaningful learning(18). In distance education, didactic material undertakes the role of guiding, since it organizes the development and dynamics of the entire teaching and learning process(14).

Thus, teaching strategies, as well as all elements that make up the didactic material, must be finalized for the beginning of the student activities, so that the student has at his/her disposal all that he/she needs so that his/her studies can be realized of autonomous form(14).

In the validation process, we know the difficulty to define the criteria for inclusion of the judges, as opposed to the dissent in the literature on specific criteria. However, the sample included experts in Urgency and Emergency, qualified and committed to academic and research activities and/or proven experience in the teaching or instruction of the BLSC(10).

The suggestions for improvements were mostly accepted and guiding the improvement of the content of the BLSC, especially related to better compression, revising references and textual quotations, performing grammar corrections related to textual agreement, necessary changes to the content proposed to be inserted in an educational technology in the modality of self-instructional distance. Besides, conciseness and clarity of digital texts are decisive elements for a pleasant and meaningful learning(19).

We should remember that the elaboration of educational objectives of the course was based on the Bloom's Taxonomy (cognitive, affective and psychomotor). Other researches also used this reference in the elaboration of assessment activities, courses and disciplines with the purpose of assessing cognitive gain in higher levels of learning(20'22).

As a theoretical framework, the AHA 2015 Guidelines Update for CPR was used, based on an international evidence assessment process that involved 250 reviewers from 39 countries and is the most used CPR protocol in the world and in Brazil(12).

Thus, in Unit I, two judges suggested to synthetize the historical evolution and another to make the chapter more dynamic to make unity more attractive.

Considering the opinion of these experts, the following question arose: how can we organize a great volume of historical information? Faced with this problem, it was found that the visualization of information through infographic plays an important role in the organization of the data, since it helps people to understand the information more easily. In addition, given the excess information to be transmitted, infographic is an approach that serves the human scale, presenting information in a contextualized, relevant and meaningful way(23).

There are a series of infographic types, of which four main types that take into account usability are highlighted, being: based on statistics, timeline, processes and location(24). Thus, the timeline infographic was the strategy chosen to adjust the content according to the request of the judges.

However, a suggestion was not adhered to this unit. This refers to the withdrawal of the evolution of resuscitation guidelines, since researchers did not find in the literature support to indicate that treating "how" and "why" the guidelines were changed every five years was reason enough to confuse the student. It is worth mentioning that Unit I is introductory and came with the concern to enable the development of subsumption to the students, in order to offer previous organizers that can serve as an anchorage for the learning of significant form of the new knowledge(13).

In Unit II, three suggestions were made. Regarding the insertion of the pulse rate assessment in infants, researchers did not find evidence that this assessment could contribute to the early onset of CPR. In addition, in the later paragraph, it is clear that in infants, CPR should be started if the heart rate is below 60 bpm/min. A study shows that even trained health professionals have difficulty confirming the presence or absence of pulse in the diagnosis of CRA and that in doubt, one should not waste time in this step by starting the CPR maneuvers(25).

One judge suggested adding in the description of chest compression techniques in infants the location of the resuscitator hands and fingers. A systematic review comparing the two techniques indicates that both are useful and that for the better understanding one must approach the step by step in BLSC courses, and thus the proposal was accepted(26).

The researchers accepted all the four suggestions for Unit III. A judge requested that the content of the item "AED in special situations" be disposed of more dynamically. Thus, a "new" form of visualization of the content, prioritizing the association of visual and textual elements, so that, the same resource of the infography of Unit I was used, condensing the content and making it more illustrative(27).

One judge suggested adding the need to remove excess medication after removal of the patch. Experimental and systematic review studies have shown the risk of severe burns in cases where AED was used in patients with a nitrate drug patch, where the patch or excess medication was not withdrawn(28).

In Unit III, it was accepted the suggestion to take systematically the use of the AED of the text body and leave it only with the proposal of the simulated video. On this issue, it is known that the redundancy of information is seen as a waste of time and can make the reading tiresome and discouraging for the student/professional(29).

In relation to Unit IV, all the suggestions made by the judges were accepted. The first suggestion was the withdrawal of the report on the discovery of the Heimlich maneuver from the text of the unit to be placed in the course library.

About this "Library" feature, AVASUS proposes this additional feature in its Manual do Conteudista (freely translated as Experts Manual)(11), in order to facilitate the student's access to complementary reading. This feature enables more dynamism, flexibility and attractiveness to the Distance Education process and are subsystems that somehow add more ease and/or satisfaction to student learning processes·30-31·.

Regarding the suggestion to include the professional presentation to the victim before the next steps to be taken, this suggestion was accepted, since the script of the simulated video was relatively technical because of the lack of experience of researchers in producing texts for video.

About this suggestion, we know that anxiety is a common feeling in the choking victim and that it can be minimized with the simple personal presentation of the professional and the brief explanation of what he/she will do to get him/her out of the imminent situation of death. This procedure is a fundamental step to be exposed in this simulated video(12).

Study limitations

As a limitation, it is worth noting that only one round of content assessment of the course conducted by the judges.

Contributions to Nursing, Health or Public Policy

We believe that articles of this nature can contribute, in particular, to the creation of strategies aimed at the prevention of diseases, as well as collaborate in the formulation of health policies aimed at strengthening the quality of care offered.

CONCLUSION

The construction of the BLSC in Distance Education was complex and challenging, since it sought to develop it with a logical-methodological coherence, within a constructivist perspective, representing an advance in the process of dissemination of the culture of the teaching of Urgency and Emergency.

As for the content validation process, six expert judges assessed the BLSC and it was verified that of the 16 suggestions made, 14 were accepted and 2 rejected. Most were related to the overall content assessment requirements, usefulness, clarity, objectivity and vocabulary.

Thus, the course had satisfactory validation (CVI - 0.94, Kappa - 0.87, 9.5 grade), and could be used to support the training of academics and professionals in BLSC.

As a limitation of the study, we highlight the short time (two months) for construction and validation determined by the AVASUS team, making it impossible to re-submit to the investigating judges.

Other validity tests could be carried out in further studies with the content already deployed in the platform, in its virtual format, with the operative part, videos and games ready to confirm its psychometric properties after its clinical application.

Sidebar

How to cite this article:

Costa IKF, Tibúrcio MP, Melo GSM, Leite JEL, Dantas RAN, Torres GV. Construction and validation of a distance Basic Life Support Course. Rev Bras Enferm [Internet]. 2018;71(Suppl 6):2698-705. [Thematic Issue: Good practices in the care process as the centrality of the Nursing] DOI: http://dx.doi.org/10.1590/0034-7167-2018-0122

CORRESPONDING AUTHOR Rodrigo Assis Neves Dantas E-mail: [email protected]

References

REFERENCES

1. Cavalcanti RLS, Monerat CAA, Barcelos LS, Oliveira RP, Silva TL. The impact of the basic distance life support course via web in the Brazilian Navy. REMPEC[Internet]. 2016[cited 2017 Dec 6];9(3):346-72. Available from: http://ensinosaudeambiente.uff.br/ index.php/ensinosaudeambiente/article/view/589/265

2. Silva AN, Santos AMG, Cortez EA, Cordeiro BC. Limits and possibilities of distance learning in continuing education in health: integrative review. Cienc Saúde Colet[Internet]. 2015[cited 2017 Dec 5];20(4):1099-107. Available from: http://www.scielo.br/pdf/ csc/v20n4/1413-8123-csc-20-04-01099.pdf

3. Silva LMG, Gutiérrez MGR, Domenico EBL. Virtual learning environment in continuing education in nursing. Acta paul Enferm[Internet]. 2010[cited 2017 Dec 7];23(5):701-4. Available from: http://www.scielo.br/pdf/ape/v23n5/19.pdf

4. Pasquali L. Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed; 2010.

5. Fehring RJ. The Fehring model. In: Carrol-Johnson RM, Paquete M. Classification of nursing diagnoses: proceeding of the tenth conference. Philadelphia: Lippincott Company; 1994.

6. Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. Taxonomia de objetivos educacionais: dominio cognitivo. Porto Alegre: Globo; 1972.

7. Ausubel DP. Aquisigão e retenção de conhecimentos. Lisboa: Plátano; 2003.

8. Randolph JJ. Online Kappa Calculator[Software][Internet]. 2008[cited 2017 Nov 21]. Available from: http://justus.randolph.name/kappa

9. Costa IKF, Medeiros LP, Dias TYAF, Salvetti MG, Dantas DV, Torres GV. Protocol validation for people with venous ulcers: a quantitative study. O Braz J Nurs[Internet]. 2016[cited 2017 Dec 6];15(2):226-35. Available from: http://dx.doi.org/10.17665/1676-4285.20165251

10. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? critique and recommendations. Res Nurs Health[Internet]. 2006[cited 2017 Dec 6];29(5):489-97. Available from: https://pdfs.semanticscholar.org/537d/5a0f099689 79b4cf4e8b0213a8f39257b393.pdf

11. Brasil. Conselho Nacional de Justiça. Centro de Formação e Aperfeiçoamento de Servidores do Poder Judiciário. Manual do Conteudista: produção de materiais didáticos para EAD[Internet]. 2016[cited 2017 Dec 6]. Available from: http://www.cnj.jus.br/ files/conteudo/arquivo/2016/01/c5d0472ec148f8a82c369007bd9fd193.PDF

12. American Heart Association - AHA. Destaques das Diretrizes da American Heart Association 2015. Atualizaçao das Diretrizes RCP e ACE. 2015[cited 2017 Dec 6]. Available from: https://eccguidelines.heart.org/wp-content/uploads/2015/10/2015-AHAGuidelines-Highlights-Portuguese.pdf

13. Moreira MA. Teorias de Aprendizagem. Sao Paulo: EPU; 2011.

14. Santos MJ, Pereira MEC, Junqueira ACV, Borba CM, Jurberg C. Reflections on the online learning of biosafety based in the meaningful learning. Cienc Cogn[Internet]. 2016[cited 2017 Dec 6];21(1):100-11. Available from: http://www.cienciasecognicao.org/revista/ index.php/cec/article/view/1096/pdf_76

15. Nobrega GAS, Souza GF, Barbosa JG, Coutinho KD, Valentim RAM. Uma análise estatística do Ambiente Virtual de Aprendizagem do Sistema Único de Saúde: Descriçao estatística dos usuários do AVASUS[Internet]. Natal: EDUFRN; 2016[cited 2017 Dec 3]:28p. Available from: http://pr.avasus.ufrn.br/local/avasplugin/dashboard/AVASUS_19out2016_impressao.pdf

16. Brasil. Ambiente virtual de aprendizagem do SUS - AVASUS[Internet|. 2017[cited 2017 Dec 4]. Available from: https://avasus.ufrn.br/

17. Bittencourt DF. As métricas, o projeto e o design na EaD. In: Correa MA. Material didático on-line da disciplina design e projetos em educaçao a distancia do curso de especializaçao em metodologia da educaçao a distancia. Florianópolis: Unisul Virtual; 2010.

18. Troncon LEA. Structuring systems for program maticassessment of medical students. Rev Brás Educ Med[Internet]. 2016[cited 2017 Dec 3];40(1):30-42. Available from: http://www.scielo.br/pdf/rbem/v40n1/1981-5271-rbem-40-1-0030.pdf

19. Linhares RN, Loureiro MJ, Ramos F, Alcantara CMG. Evaluation of teaching digital technologies: Brazilian and Portuguese indicators. Estud Aval Educ[Internet]. 2017[cited 2017 Dec 6];28(67):12-31. Available from: http://dx.doi.org/10.18222/eae.v0ix.4303

20. Zaidi NB, Hwang C, Scott S, Stallard S, Purkiss J, Hortsch M. Climbing Bloom's taxonomy pyramid: lessons from a graduate histology course. Anat Sci Educ[Internet]. 2017[cited 2017 Dec 4];10:456-64. Available from: https://doi.org/10.1002/ase.1685

21. Choudhury B, Gouldsborough I, Shaw FL. The intelligent anatomy spotter: a new approach to incorporate higher levels of Bloom's taxonomy. Anat Sci Educ[Internet]. 2016[cited 2017 Dec 4];9(5):440-5. Available from: https://doi.org/10.1002/ase.1588

22. Morton DA, Colbert-Getz JM. Measuring the impact of the flipped anatomy classroom: the importance of categorizing an assessment by Bloom's taxonomy. Anat Sci Educ[Internet]. 2017[cited 2017 Dec 4];10(2):170-75. Available from: https://doi.org/10.1002/ase.1635

23. Meireles AV, Camara RJ, Souto VT. Organizando big data gerado colaborativamente:um estudo de caso do Infográfico do tipo Linha do tempo do Marco Civil da Internet. Rev Design Tecnol Soc[Internet]. 2016[cited 2017 Dec 2];2(1):105-19. Available from: http://periodicos.unb.br/index.php/design-tecnologia-sociedade/article/view/19965/14154

24. Siricharoen WV. Infographics: the new communication tools in digital age. The International Conference on E-Technologies and Business on the Web (EBW2013)[Internet]. 2013[cited 2017 Dec 3];169-74. Available from: https://www.researchgate.net/ publication/256504128_Infographics_The_New_Communication_Tools_in_Digital_Age

25. Hategeka C, Shoveller J, Tuyisenge L, Kenyon C, Cechetto, DF, Lynd LD. Pediatric emergency care capacity in a low-resource setting: an assessment of district hospitals in Rwanda. PLos ONE[Internet|. 2017[cited Dec 7 2017];12(3):e0173233. Available from: https://doi.org/10.1371/journal.pone.0173233

26. Douvanas A, Koulouglioti C, Kalafati M. A comparison between the two methods of chest compression in infant and neonatal resuscitation: a review according to 2010 CPR guidelines. J Matern Fetal Neonatal Med[Internet]. 2017[cited 2017 Dec 6];5:1-12. Available from:https://www.ncbi.nlm.nih.gov/pubmed/28282762

27. Menezes HF, Queiroz JER. Análise dos principios visuais no projeto gráfico: estudo de caso envolvendo infográficos jornalísticos. Blucher Design Proceedings[Internet]. 2016[cited 2017 Dec 6];2(9):4673-85. Available from: http://www.proceedings.blucher. com.br/article-details/anlise-dos-princpios-visuais-no-projeto-grfico-estudo-de-caso-envolvendo-infogrficos-jornalsticos-24638

28. Lampert A, Seiberth J, Haefeli WE, Seidling HM. A systematic review of medication administration errors with transdermal patches. Expert Opin Drug Saf[Internet]. 2014[cited 2017 Dec 5];13(8):1101-14. Available from: https://www.ncbi.nlm.nih.gov/ pubmedhealth/PMH0068622/

29. Escrivao G, Nagano M. Knowledge management in environmental education: case studies in environmental education programs in Brazilian universities. Perspect Cienc Inf[Internet]. 2014[cited 2017 Dec 4];19(4):136-59. Available from: http://www.scielo.br/ pdf/pci/v16n1/a06v16n1

30. Correa MA. Os materiais didáticos como recursos fundamentais de potencializaçao da qualidade do ensino e aprendizagem na EAD. Rev E-Tech[Internet]. 2013[cited 2017 Dec 5];6(1):125-40. Available from: https://doi.org/10.18624/e-tech.v6i1.280

31. Carbogim FC, Oliveira LB, Mendonça ET, Marques DA, Friedrich DBC, Püschel VAA. Teaching critical thinking skills through problem based learning. Texto Contexto Enferm[Internet]. 2017[cited 2018 Feb 18];26(4):e1180017. Available from: http://www.scielo.br/pdf/tce/v26n4/en_0104-0707-tce-26-04-e1180017.pdf