AN EXPERIMENTAL ANALYSIS OF COBOL PROGRAM QUALITY THROUGH THE APPLICATION OF SOFTWARE METRICS TO PROGRAMS WRITTEN WITH AND WITHOUT STRUCTURED PROGRAMMING TECHNIQUES

Since the early 1970s, most discussions of software development methodology have focused on structured programming techniques. Supporters of structured programming have uniformly claimed that the techniques they advocate produce software of high quality (as defined by readability, modifiability, and verifiability) than that developed using traditional approaches. To date, verification of the claim that structured programming improves software quality has been based primarily on ex post facto studies.

The objective of this research was to evaluate the claims of structured programming through a controlled experiment in which a set of software metrics would be applied to the same computer programs generated by individuals with and without exposure to structured programming. The selected metrics evaluated quality in terms of readability, modifiability, and verifiability.

University students in a beginning COBOL programming course were divided into control and experimental groups based on aptitude and background. After learning COBOL fundamentals, the experimental group was exposed to multi-media materials on structured programming, while the control group was assigned commensurate reading materials on programming style without a structured focus.

Source program listings for course assignments completed by both groups were scored by outside evaluators trained in the application of metrics relating to program readability and modifiability. These were supplemented with measures (objective counts) related to these same attributes and to verifiability. All evaluators were unaware of the experimental design. Evaluator worksheets were coded by group membership, keypunched, and analyzed by SPSS. T-tests of the sixteen measures (twelve individual metrics, three subtotals, and one overall total) yielded seven which showed the experimental group to be superior in program quality and only one that showed the control group to be superior at the .90 confidence level. These findings led to rejection of the primary null hypothesis which postulated that experimenal group programs were not of demonstrably higher quality than control group programs. Additional statistical measures (discriminant analysis and correlation) were applied to provide added insights into the data.