Human vs Computer aided testing (C.A.T.) Results at Department of Anatomy, Histology and Forensic Medicine, University of Florence

Headnote

Key words

Computer aided test, Human Anatomy software

At the Anatomical section of the Department of Anatomy, Histology and Forensic Medicine, University of Florence, a software for the administration of multiple-choice tests has been designed and developed in 2002.

The program, called ATOL (Anatomy Test on Line), allows the integration, by web interface, into the same system of both the verification phase of learning (to be performed in controlled site with a personalized and unique profile) and the phase of remote self testing. The questions may be coupled to images.

ATOL was used since May 2002 to December 2008 as a tool for exams concerning the Locomotor and Circulatory Systems for students attending the first and second year of Medical School.

1990 students were subjected to the exam concerning the Locomotor system, 1678 (88.8%) successfully, with an average score of 24.02±3.03 (mean±standard deviation).

Starting from 2008/09 it was decided to go back to the oral exam. In recent sessions the ATOL tests on Locomotor was administered to a large cohort of students as a moment of self-evaluation, one day before the oral exam.

This study presents the comparison between the results of 165 students (first year year of Medical School) who have been tested, almost simultaneously, by written and oral exam.

Through statistical analysis, carried out with Origin 8 Pro (OriginLab), we examined whether a significant difference existed between scores obtained by CAT and traditional examination and if it was possible to find a correlation between the results obtained whit the two methods of examination.

For the evaluation of the results, analysis of variance (ANOVA validated method of Bonferroni's analysis of the means) and determination of Pearson's coefficients were applied.

In summary a clear difference exists between the groups Written test - Oral exam (mean score±standard deviation 23,6±3,1 vs 28,6±2,4; p = 9,7 x 10-7) but the ATOL score and the subsequent oral exam score are also strongly correlated (rp = 0.52, p = 1,5 x 10-12).

These results indicate that the scores between the two methods, different and correlated, are a reasonable indication of the actual preparation of the examined students.

The highest, on average, evaluation obtained by traditional examination is probably linked to the traditional teacher-learner interactions that take place during the exam and that generally direct to the correct answer, what usually does not happen with the use of CAT.