Content area
This study aims to determine the operating elements of various levels of the Capability Maturity Model (CMM®) and important indicators of organizational performance (IOPs) in the software industry. The various levels of CMM act as critical factors (CFs), which are the input variables necessary to implement CMM. IOPs are the output variables on which the effect of implementing CMM is measured. The current levels of presence of the CFs and IOPs and the changes in the CFs and IOPs due to CMM implementation were determined in this study using two types of scales, namely, level scale and change scale, in order to measure (or capture) and analyze the perception of respondents in this research. Many interesting and valuable insights are provided. It has been found that most of the firms find it easier to set standards than to improve further. Moreover, financial gains are not immediate, and firms must strive to continuously improve in order to gain financially from CMM implementation. The study also revealed that CMM implementation has a significant positive change on organizational performance. [PUBLICATION ABSTRACT]
Full text
This study aims to determine the operating elements of various levels of the Capability Maturity Model (CMM®) and important indicators of organizational performance (IOPs) in the software industry. The various levels of CMM act as critical factors (CFs), which are the input variables necessary to implement CMM. IOPs are the output variables on which the effect of implementing CMM is measured. The current levels of presence of the CFs and IOPs and the changes in the CFs and IOPs due to CMM implementation were determined in this study using two types of scales, namely, level scale and change scale, in order to measure (or capture) and analyze the perception of respondents in this research. Many interesting and valuable insights are provided. It has been found that most of the firms find it easier to set standards than to improve further. Moreover, financial gains are not immediate, and firms must strive to continuously improve in order to gain financially from CMM implementation. The study also revealed that CMM implementation has a significant positive change on organizational performance.
Key words: capability maturity model, indicators of organizational performance maturity levels
INTRODUCTION
The software industry has acclaimed quality as a key factor that helps organizations to achieve success and a competitive edge in global markets. Many organizations have applied quality tools and techniques that they employed in the other departments to their software department. Software defects and repairs cost a fortune if not detected early. Thus, quality assurance is a serious issue in software development. There is an implied need to have procedures that help ensure defect-free software.
In software, the lack of "bugs" in a product is recognized as product quality, though there is no unique definition for software quality and its domain is hazy. The idea of quality in an intangible product like software can be difficult to define and measure as opposed to defining quality in the manufacturing and service sectors. The general agreement, however, is that there is a greater need to respond to customer needs and assure quality of processes in complex software development projects than there is to define and measure quality.
Once a basic quality control level has been achieved in the work processes by satisfying quality standards, most companies aim to achieve "worldclass" status by developing company processes. Thus, quality becomes an integral part of software processes. The measure of the capability of a process to repeatedly reproduce outputs is called its control status, and it is used as an indicator of the quality of a software process. The most important capability maturity model is the Software Engineering Institute's (SEI) five-level model. It provides a specific approach for practicing total quality management (TQM) in the software industry. Although recent, the framework has been adapted by the French and the U.S. ministries of defense (Heinze 2002).
The CMM was developed by the software community with stewardship provided by the SEI of Carnegie Mellon University. It is a model for judging the maturity of the software development processes of an organization and for identifying the key practices that are required to increase their maturity. It outlines the principles underlying software process maturity and provides organizations with an evolutionary path to improve in stages toward continuous process improvement.
A maturity level is a well-defined evolutionary plateau toward achieving a mature software process. Each maturity level comprises a set of process goals that, when satisfied, stabilize an important component of the software process. This results in an increase in the organization's software process capability.
As a software organization gains in software process maturity, it builds an infrastructure and a corporate culture that supports the methods, practices, and procedures of the business. This eventually helps the organization endure, even after those who originally defined those methods leave the organization. As the organization's software process matures, the difference between targeted results and actual results decreases across projects.
THE FIVE LEVELS OF SOFTWARE PROCESS MATURITY IN THE CMM
The CMM provides a framework for achieving continuous process improvement through five evolutionary levels. These five maturity levels enable an organization to measure its maturity stage and evaluate its software process capability. The levels also help an organization prioritize its improvement efforts. The five levels of process maturity and process capability obtained at each level are shown in Figure 1.
Level 1-The Initial Level
At the initial level, the organization typically does not have a stable environment for developing and maintaining software. During a crisis, projects do not operate on planned procedures, and revert to coding and testing. Success depends on having an exceptional manager, and a seasoned and effective software team. When these people leave the project, their stabilizing influence leaves with them. Thus, at level 1, software capability is more a characteristic of the individuals and not of the organization.
Level 2 -The Repeatable Level
At the repeatable level, policies for managing a software project and procedures to implement the policies are established. Software project standards are defined, and the organization ensures that the standards are faithfully followed. Realistic project commitments are based from the results observed on previous projects and on the requirements of the current project. Further, the costs, schedules, and functionality for a project are tracked. The software process capability of level 2 organizations can be summarized as "disciplined."
Level 3-The Defined Level
At the defined level, project staff adapt the organization's standard software process to develop their own defined software processes, which account for the unique characteristics of the project. An organizationwide training program is implemented to ensure that the staff and managers have the knowledge and skills required to fulfill their assigned roles. Because the software process is well defined, management has good insight into technical progress on all projects. Thus, the software process capability of level 3 organizations can be summarized as "standard and consistent."
Level 4-The Managed Level
At the managed level, the organization sets quantitative quality goals for both software products and processes. Projects achieve control over their products and processes by narrowing the variation in their process performance to fall within acceptable quantitative boundaries. When the known limits of the process are exceeded, actions are taken to correct the situation. This produces software products that are of predictably high quality. Thus, the software process capability of level 4 organizations can be summarized as "quantifiable and predictable."
Level 5-The Optimizing Level
At the optimizing level, the entire organization is focused on continuous process improvement. Here, the causes of defects are identified, known types of defects are prevented from recurrence, and lessons learned are disseminated to other projects. Improvements in software processes occur both by incremental advancements in the existing processes, as well as by innovations using new technologies and methods. The software process capability of level 5 organizations can be characterized as "continuously improving."
Each maturity level is composed of several key process areas (KPAs). Each KPA is organized into five sections, called "common features" that specify key practices that, when collectively addressed, accomplish the goals of the KPA. The path to achieving the goals of a KPA may differ across projects based on differences in application domains or environments. The maturity levels in CMM and their KPAs are shown in Figure 2. For a detailed description of each of these KPAs, see the SEI CMM Web site at www.sei.cmu.edu/cmm/cmms/cmms.html.
Carroll (1995) found that TQM could be used to integrate research activities and actual practices in software development in many software areas such as metrics, process improvement, design, and so on. Barnett and Raja (1995) researched the application of quality function deployment (QFD) to software development. They opined that many quality tools and techniques had been applied to software, as in the manufacturing industry, without understanding the uniqueness of software products and processes. Further, they proposed a four-stage model for software QFD. The requirements were based on customer segments contributing to the value of organizational processes. The model linked product design to business processes, and it could serve as a formal method of obtaining system requirements that are supportive of enterprise priorities.
Harter, Krishnan, and Slaughter (2000) empirically investigated the relationship between the development effort and variables such as process maturity measured on the CMM-maturity scale, product quality, and development cycle time. They found that higher levels of process maturity lead to higher quality, and result in reduced cycle time and developmental efforts. An organization that has acquired CMM level 5 is expected to maintain very high quality standards. Jalote (2000) found the CMM to be a widely used framework for quality management in software industries.
Parzinger and Nath (2000) examined the relationship between TQM practices and software quality in the U.S. software industry. They used employee empowerment, quality measures, executive commitment, general training methods, customer needs assessment, process evaluation, specific skills training, and cycletime reduction as the TQM implementation factors and customer satisfaction, level of CMM certification, cost of quality, and compliance with ISO 9000-3 as the dimensions of software quality. They concluded that overall success was significantly determined by employee empowerment, customer needs assessment, and specific skills training. Yang (2001) proposed an eight-step approach for the fulfillment of ISO 9000 requirements in the software industry. It was suggested that becoming ISO 9000 registered was the only first step to achieving consistent software quality.
Li, Chen, and Lee (2002) conducted a nationwide study on Taiwanese software companies, most of which were moving from the "initial" level to the "repeatable" level in CMM classification. A major finding of the study was that the threshold-based CMM assessment method was not feasible for identifying the overall software process management infrastructure of a business. Further, the percentage of key practices implemented by the companies was very low, which could be attributed to the lack of adequate training and software engineering knowledge among software professionals. Moreover, companies were poor in gathering nonvalue-added supporting activities such as code and test statistics. Finally, empowerment should happen to everyone in the company.
Isaac, Rajendran, and Anantharaman (2003) developed a conceptual framework for software quality from the client's perspective, and they found that product quality, client focus, process quality, operational effectiveness, employee competence, infrastructure, and facilities were the determinants of software quality. Niazi, Wilson, and Zowghi (2003) discovered that advances in the implementation of software process improvement standards did not match the advances in the development of those standards, resulting in the lack of success of such initiatives. So, they came up with a framework to provide an effective strategy for implementing software process improvement initiatives. Hence, it could be used effectively to design similar initiatives. Agrawal and Chari (2007) studied the impact of process maturity on effort, quality, and cycle time. They determined that high levels of process maturity (indicated by CMM level 5 certification) reduced the variance of software development outcomes that were caused by factors other than software size.
Considering the growth of the software industry (globally as well as in India) and the increasing affinity toward acquiring quality certification by software organizations, there is a need to explore the relationship between quality certification, particularly CMM certification and organizational performance, in the software industry. The current research was conducted with the objective of filling this gap.
Findings from the Literature Review
Despite many works on definitions and perceptions of quality, only a few studies in the software industry have dealt with quality practices such as the effort in implementing CMM (Harter, Krishnan, and Slaughter 2000) and the effect of CMM certification (Parzinger and Nath 2000; Li, Chen, and Lee 2002). There is a need to further explore the various levels of CMM certification and their effect on organizational performance.
RESEARCH OBJECTIVES
The objectives of this study are to:
1. Determine the levels of presence of and changes in the critical factors (CFs) and indicators of organizational performance (IOPs) in a sample of CMM certified firms.
2. Find the influence of implementing CMM on organizational performance.
The CFs are essentially "input" variables that are within the overall control of the organizations. In contrast, the IOPs can be regarded as "output" variables that describe the performance of the firms on various aspects, particularly before and after CMM implementation. These variables were selected following the literature review and discussions held with executives in the sample firms. To achieve these objectives, a survey (presented in the Appendix) was conducted from a sample of CMM certified firms to gather practitioners' perceptions on various aspects of CFs and organizational performance. The responses and findings can be effectively used for improving quality programs in the industry. A detailed discussion on CFs and IOPs is presented later.
CRITICAL FACTORS FOR FULFILLING THE CMM
An objective of the study is to identify the major operating elements of CMM standards in the software industry, based on an empirical analysis. Combinations of these operating elements are termed KPAs in CMM. These elements that constitute various CMM levels represent the CFs. Even though there are five levels specified in the CMM standards, essentially there are only four levels, as all software organizations, whether going for CMM certification or not, are considered to be at level 1. Therefore, obtaining CMM certification is necessary to achieve levels higher than level 1 (level 2 to level 5). As a result, level 1 organizations are not included in this study. The five levels were described earlier and are denoted as L-I, 1-2, L-3, L-4, and L-5, respectively, in the study. The operating elements of the five levels of CMM, as identified from the SEI's manual, are given in Appendix 1. There is no literature provided for CFs of CMM, as they are these levels derived from the SEI's manual.
INDICATORS OF ORGANIZATIONAL PERFORMANCE
All the efforts, including quality management practices, are futile if they do not contribute to the firm's bottom line. So, it is essential to link quality practices with organizational performance. Flynn, Shroeder, and Sakakibara (1994) emphasized distinguishing quality management practices (inputs) from quality performance (outputs). Samson and Terziovski (1999) found that TQM practices explained a significant portion of variance in firm performance. They used measures such as employee morale, customer satisfaction, cost of quality, productivity, and so on as indicators of organizational performance. It was revealed that behavioral factors like leadership, people management, and customer focus predicted performance more significantly than operational factors such as process improvement, benchmarking, information and analysis, and so on. Terziovski and Samson (1999) studied the link between TQM practices and organizational performance. The study concluded that TQM had a significantly positive effect on organizational performance. Agus, Krishnan, and Kadir (2000) investigated the impact of TQM practices on the financial performance of the firms. They found that TQM practices significantly impacted financial performance through customer satisfaction. They operationalized financial performance through overall financial performance, profitability, and revenue growth. Sureshchandar, Rajendran, and Anantharaman (2003), in their study on total quality service in Indian banks, used customer focus and employee satisfaction as the measures of quality performance. Hence, it would be worthwhile to investigate if quality efforts expended in CMM certification really contribute to firm performance.
Performance indicators of a firm quantitatively represent the various firm- and market-related aspects of its products, services, resources, and productivity. They provide data about how well a company performs, potential areas of improvement, and the gap between the firms' results and goals, and they are used to determine if the firm's processes are under control and if customers are satisfied. From the literature review, widely used performance indicators that satisfy these objectives were selected for this study. The IOPs considered in this study are:
* Customer satisfaction
* Employee morale
* Profitability
* Overall productivity
* Reduction in quality costs
* Overall financial performance
* Overall operational performance
These indicators are described, along with the studies in which they have been used, as follows.
Customer Satisfaction (CS)
The very definition of quality involves meeting customers' expectations. Further, satisfaction of customers adds to the competitive advantage of an organization. According to W. Edwards Deming (1986), a firm should not only focus on "do it right first time," but also delight its customers to improve its business performance. Customer satisfaction was used as an indicator of both organizational performance (Terziovski and Samson 1999) and operational performance (Samson and Terziovski 1999) of an organization. Pun (2001) considered customer satisfaction as a performance indicator of TQM efforts.
Employee Morale (EM)
A conducive work environment is essential for effective employees. The work environment influences the employees' abilities to improve the quality of work (Issac, Rajendran, and Anantharaman 2004). Employees are the assets of an organization, and their morale and motivation lead to improved customer focus, resulting in customer satisfaction. This, in turn, leads to an increase in market share and competitiveness. Terziovski and Samson (1999) considered employee morale as a measure of organizational performance.
Profitability (Prf)
Profitability indicates the efficiency of a company at generating earnings. The primary goal of any business is to make a profit. Hence, profitability also implies how well a firm performs. Agus, Krishnan, and Kadir (2000) included profitability as a performance indicator of an organization in their study on TQM. Pun (2001) also considered profitability as a measure of a firm's performance.
Overall Productivity (OP)
Overall productivity indicates if the firm effectively uses its resources to meet its objectives and goals. Terziovski and Samson (1999) and Pun (2001) included productivity as an important measure of organizational performance.
Reduction in Quality Costs (RQC)
Cost of quality, as addressed by Taguchi, is an important measure of organizational performance. It is necessary to execute cost-effective ways to improve the quality of products, services, and processes. Terziovski and Samson (1999) included cost of quality as a measure to indicate organizational performance.
Overall Financial Performance (OFP)
While profitability gives an indication of a firm's business performance, overall financial performance provides information about how well an organization meets its (financial) goals, and how effectively it uses resources to achieve an efficient process that results in good products/services. Agus, Krishnan, and Kadir (2000) considered overall financial performance as an important indicator of organizational performance.
Overall Operational Performance (OOP)
Operational performance is the practice of understanding, optimizing, and aligning the operational business activities and processes to a common set of goals and objectives to improve effectiveness. The ability to deliver operational performance excellence in the areas of core competency is one key to success. Terziovski and Samson (1999) considered operational performance as an essential measure of organizational performance.
THE SAMPLE AND THE INSTRUMENT FOR THE CMM
One of the main objectives of this study is to validate through empirical analysis the CFs identified for enabling CMM. Only those firms that meet the following criteria were included in the study:
* The firm belongs to the software industry.
* The firm is certified CMM (any level, from level 2 to level 5).
As there appears to be no prior instrument available to measure the CFs for CMM in the software sector, an instrument was developed. To construct the sampling frame, the study used various sources of information about companies, including information regarding the respondents as well as the firms.
The units of analysis in this study are CMM certified firms. From the NASSCOM Web site (see www.nasscom.org), 82 CMM certified software firms were randomly chosen. Questionnaires were mailed to personnel who were responsible for implementing CMM in their respective organizations. After persistent attempts spanning more than six months, 33 CMM certified firms responded.
The questionnaire consisted of a general section to obtain information about the respondents' organizations, and two other Sections, Section A and Section B. The questions in Section A and B pertained to the CFs and IOPs, respectively. section A included 40 items indicating the operating elements. Examples of operating elements include:
* Presence of reaction-driven systems rather than effectively planned systems
* Level of adherence to commitments that are made after proper analysis of projects
* Integration of software engineering and management activities into coherent, well-defined software processes
* Application of a comprehensive measurement program to the software work products
* Ability to focus on performing innovation efficiently in a dynamic environment
The operating elements were identified from the manual for CMM standards. In effect, each CF corresponds to a "maturity level," with the constituent operating elements being the "key process indicators."
In this study, two seven-point scales were used in the questionnaire. The choice of seven-point scales is consistent with the literature on TQM and ISO systems (Huarng 1998; Sureshchandar, Rajendran, and Anantharaman 2001). The first one, ranging from O to 6, was used to indicate the perceived level of presence of various operating elements of CMM. The scale is given below:
As this scale includes the entire range for the level of current presence of an operating element, it is called the level scale.
The second scale, ranging from -3 to +3 through O, is used to indicate the extent of change in an operating element over a period of time, because of the implementation of CMM. This scale is given below:
As this scale indicates the extent of change in the operating elements of CMM, it is called the change scale. The same seven-point scales were used to measure the IOPs, and measurement of all IOPs on these scales was done to maintain uniformity, even though for some indicators like "profitability" actual company data could have been used.
DESCRIPTIVE STATISTICS OF THE CRITICAL FACTORS
The descriptive statistics of the CFs for CMM, namely, minimum, maximum, mean, and standard deviation (SD) of the CF scores (in the level and change scales) are presented and discussed. Box and whisker plots are used to indicate the interquartile range (IQR), outliers (shown by [white circle]), and extreme values (shown by *) for each CF.
Analysis of Results-Critical Factors: Level Scale
This section presents the analysis of results (see Table 1 and Figure 3), for the CFs measured on the level scale.
From Table 1 and Figure 3, the following are the observations with respect to levels of presence of CFs.
* L-1 has the lowest mean (value of 2.57, indicating a level of presence between "moderate" and "significant" on the seven-point level scale). This shows that firms do not have most of the operating elements of level 1. This is logical, as level 1 characterizes a firm in its rudimentary state of process maturity and hence not certified with higher levels of process maturity. In other words, as the survey includes only certified firms, operating elements characteristic of level 1 (or noncertified firms) are expected to be absent. Further, it has the highest SD value. This shows that the respondents' opinions about the KPAs of L-1 are widely dispersed.
* L-2 has the second lowest mean (value of 4.68, indicating a level of presence between "high" and "very high" on the seven-point level scale). This implies that firms give less attention to the operating elements in level 2. Further, no responses were obtained from level 2 firms, which might have led to this low mean value. As it has the lowest SD and the lowest range, it is evident that responses for L-2 are more or less uniform.
Table 1 Descriptive statistics for the CFs-level scale.Figure 3 Box and Whisker plots for the CFs-level scale.
* L-3 has the highest mean (value of 5.03, implying "very high" level of presence on the seven-point level scale). This is because most firms focus on the KPAs of level 3.
* L-4 has the second highest mean value (of 4.87, closer to "very high" level of presence on the seven-point level scale), from which one can infer that firms give a higher focus toward the operating elements of L-4. The highest range value for L-4 may be attributed to the two extreme values on the lower side (respondent numbers 5 and 26).
* L-5 has a relatively low mean value (value of 4.73, indicating a level of presence between "high" and "very high" on the seven-point level scale), which shows that firms have not yet achieved the higher levels of presence of operating elements of L-5, as these elements are hard to accomplish in the short run.
The CMM companies scored a maximum mean value with respect to level 3, which means that they are performing well in level 3 KPAs. This is so because most of the firms find it easier to set standards than to improve further.
Analysis of Results-Critical Factors: Change Scale
This section presents the analysis of results (shown in Table 2 and Figure 4) for the CFs measured on the change scale. From Table 2 and Figure 4, the following are the observations with respect to changes in CFs.
* L-1 has the lowest mean value of 0.25 (implying "no change" on the seven-point change scale). This indicates that firms do not give importance to KPAs characterizing ad-hoc processes and hence have minimal change in L-I. Further, it has the highest SD, IQR, and range values. This may be attributed to the pair of outliers and extreme values (respondent numbers 25 and 12, and 23 and 19, respectively) in the responses.
* L-2 has a relatively low mean value (of 1.81, indicating a positive change between "marginal" and "significant" on the seven-point change scale). Since there are no L-2 certified firms in the sample, it is natural that the change in L-2 is less. Despite the presence of outliers (respondent numbers 8,21, 23, and 25), it has the lowest SD, IQR, and range values, which shows that respondents have similar perceptions of the KPAs in L-2.
* Firms have undergone the maximum change in L-3 (with a mean value of 2.03, implying a "significant" positive change on the seven-point change scale), which implies that as firms realize the importance of the KPAs denoting this level, they begin to provide more attention to them. The moderate SD, IQR, and range values might be due to outliers (respondent numbers 8,23, and 25).
* L-4 has the second highest mean (1.94, a value closer to "significant" positive change on the seven-point change scale), which indicates that firms have experienced higher change in the operating elements of L-4. Firms, because of the process of certification, realize the importance of KPAs of L-4 and strive to improve upon these KPAs. It has moderate SD, IQR, and range values, which might be due to an outlier (respondent number 25).
Table 2 Descriptive statistics for the CFs-change scale.Figure 4 Box and Whisker plots for the CFs - change scale
* L-5 has the second lowest mean value (of 1.80, indicating a positive change between "marginal" and "significant" on the seven-point change scale). Firms find it hard to improve the KPAs of L-5, as it has very stringent requirements of process maturity. It has moderate SD, IQR, and range values, which might be due to an outlier (respondent number 25).
The CMM companies have scored a maximum change with respect to level 3, in line with the authors' earlier findings. This indicates that most firms focus on level 3, on the way to the implementation of CMM.
DESCRIPTIVE STATISTICS OF THE INDICATORS OF ORGANIZATIONAL PERFORMANCE
The descriptive statistics for the IOPs of CMM, namely, minimum, maximum, mean, and SD, are presented. Further, Box and Whisker plots are used to indicate the IQR, outliers, and extreme values.
Analysis of Results-Indicators of Organizational Performance: Level Scale
This section presents the analysis of results (shown in Table 3 and Figure 5) for the IOPs measured on the level scale. From Table 3 and Figure 5, the following are the observations with respect to levels of presence of IOPs.
* CS has the highest mean value (of 5.33, indicating a level of presence between "very high" and "complete" on the seven-point level scale) and the lowest SD and range values. Most firms agree that as their maturity levels increase, the process capability increases, resulting in increased customer satisfaction.
* EM has a relatively higher mean value (of 4.93, indicating a level of presence between "high" and "very high" on the seven-point level scale). This shows that firms focus on their internal customers also, with an increase in their process maturity. Despite the presence of extreme values (respondent numbers 12 and 20), it has low IQR and minimum range values.
* Prf has a relatively low mean value (of 3.26, indicating a level of presence between "significant" and "high" on the seven-point level scale). As the primary objective of the firms is to improve process maturity and process capability, they might obtain higher profits only in the long run.
* OP's mean is relatively higher (a value of 5.00, equal to "very high" level of presence on the seven-point level scale). As the EM and CS are higher, the resources turn more productive. Despite the presence of two extreme values respondent numbers 22 and 32), OP has the lowest IQR and range values, which indicates that the respondents have similar opinions about OP.
Table 3 Descriptive statistics for the IOPs - level scale.Figure 5 Box and Whisker plots for the IOPs- level scale
* RQC's mean value (of 4.64, indicates a level of presence between "high" and "very high" on the seven-point scale) is relatively low. It is not easy for the firms to reduce quality costs initially. Further, the range value for RQC is the maximum (which might be due to the extreme value-respondent number 30), which shows that respondents have widely dispersed perceptions about RQC.
* OFP has a relatively low mean value of 3.53 (indicating a level of presence between "significant" and "high" on the seven-point level scale), as it would take some time for the firms to perform better financially. The highest range value may be due to the presence of an outlier (respondent number 20).
* OOP has a relatively higher mean value (of 4.86, closer to "very high" level of presence on the seven-point level scale). This may be due to the higher level of presence of OP. The lowest IQR shows that responses are not dispersed. The highest range might be due to the presence of three extreme values (respondent numbers 29,20, and 23).
The CMM companies have scored higher values (measured in the proposed level scale) with respect to operational indicators (namely, OP, RQC, and OOP) than financial indicators. This means that financial gains are not immediate (unlike the operational gains), and firms have to strive to continuously improve to gain financially. Further, CS and EM are high, indicating that firms focus both on internal and external customers.
Analysis of Results-Indicators of Organizational Performance; Change Scale
This section presents the analysis of results (presented in Table 4 and Figure 6) for the IOPs, measured on the change scale. From Table 4 and Figure 6, the following are the observations with respect to changes in IOPs.
* CS has a relatively high mean value (of 2.18, closer to "significant" positive change on the seven-point change scale) and the lowest SD, IQR, and range values. This is because firms unanimously begin to focus more on their customers because of certification.
* EM's relatively higher mean value of 2.27 indicates a positive change between "marginal" and "significant" on the seven-point change scale. This shows that firms not only focus on their external customers but also on their internal customers.
* Prf has the lowest mean value (of 0.97, almost equal to "marginal" positive change on the seven-point change scale) and the highest SD, IQR, and range values. This shows that because of certification, firms do not experience an appreciable increase in their profits in the short term.
* OP has a relatively low mean value of 1.76, indicating a positive change between "marginal" and "significant" on the seven-point change scale. Though firms strive to improve their productivity by increasing their maturity levels, it cannot be realized in the short run.
Table 4 Descriptive statistics for the IOPs-change scale.Figure 6 Box and Whisker plots for the IOPs-change scale
* RQC's moderate mean value (of 1.87, closer to "significant" positive change on the seven-point change scale) shows that because of certification, firms try to reduce the costs of improving their process maturity. The highest IQR and range values imply that respondents' perceptions about RQC are widely dispersed.
* OFP has a mean value of 1,29, indicating a positive change between "marginal" and "significant" on the seven-point change scale. OFP, being a longterm benefit, has undergone relatively low changes in the firms. The moderate SD and maximum range values make it evident that respondents have varying perceptions about OFP in their firms.
* OOP has a high mean value (of 2.28, indicating a positive change between "significant" and "complete" on the seven-point change scale) and the highest range values. Firms accomplish an improvement in their internal operations because of certification. This might be attributed to higher changes in EM and RQC. It has high SD and the maximum range values, which might be due to the presence of an outlier (respondent number 29).
In terms of the change scale, it is evident that firms have undergone larger changes with respect to operational and soft (human resources related) indicators, similar to the authors' earlier findings. Hence, it is clear that firms gain in respect of these operational and soft indicators within a shorter time, compared to the financial gains.
EFFECT OF IMPLEMENTATION OF THE CMM ON ORGANIZATIONAL PERFORMANCE
To see if the implementation of CMM has "desirable effects" on organizational performance, measured by IOPs, the mean values of the IOPs were compared with the test value of zero on the change scale. The test value zero indicates that there is no significant change in the respective IOPs because of CMM implementation.
To find out if CMM implementation has a significant positive influence on the IOPs, the following hypothesis was framed and tested with respect each IOP (that is, customer satisfaction, employee morale, profitability, overall productivity, reduction in quality costs, overall financial performance, and overall operational performance).
* H^sub 0^: There is no significant change in the IOPs due to the implementation of CMM.
From Table 5, it is seen that there is a significant change at the 0.01 level (rejection of HO) with respect to each IOP. Further, the mean values of all measures are positive, indicating that there is an improvement with respect to all the IOPs after implementing CMM. This shows that the implementation of CMM improves the overall organizational performance.
SUMMARY AND CONCLUSIONS
This study dealt with some aspects of CMM implementation in Indian software firms. Two sets of variables were identified. These are CFs that enable organizations to fulfill the CMM standards and IOPs. A questionnaire was developed to identify the operating elements of CMM and validated. Further, the levels of presence of CFs and IOPs after certification, and the changes in the CFs and IOPs due to certification, were presented. It is seen that firms have obtained maximum scores in level 3. This means that during the certification process, the most important benefit that firms obtain is getting their processes standardized. It is also evident that CMM implementation improves the overall organizational performance, thereby bringing out the importance of CMM certification.
The CMM companies have scored higher values (measured in both the proposed level scale and change scale) with respect to operational indicators (namely, overall productivity, reduction in quality costs, and overall operational performance) and soft indicators (namely, customer satisfaction and employee morale) than the financial indicators. This means that financial gains are not immediate, unlike operational and soft gains, firms must strive to improve continuously to gain financially, and firms should not suspend their quality efforts by measuring the performance in the short run, as financial gains could be realized only in the long run. Future studies can use a larger sample size to determine the relationship between the CFs and organizational performance through correlation and regression analyses.
ACKNOWLEDGMENT
The authors are grateful to the reviewers and the editor for the suggestions and comments to improve the earlier versions of this article.
REFERENCES
Agrawol, M., and K. Chari. 2007. Software effort, quality and cycle time: A study of CMM level 5 projects. IEEE Transactions on Software Engineering 33: 145-156.
Agus, A., S. K. Krishnan, and S. L. S. A. Kadir. 2000. The structural impact of total quality management on financial performance relative to competitors through customer satisfaction: A study of Malaysian manufacturing companies. Total Quality Management 11:5808-5819.
Barnelt, W. D., and M. K. Raja. 1995. Application of QFD to the software development process. International Journal of Quality and Reliability Management 12: 2442.
Carroll, J. 1995. The application of total quality management to software development. Information Technology and People 4: 3547.
Deming, W. E. 1986. Out of the crisis. Cambridge, Mass.: MIT Center for Advanced Engineering Study.
Flynn, B. B., R. G. Schroeder, and S. Sakakibara. 1994. A framework for quality management research and an associated measurement instrument. Journal of Operations Management 11: 339-366.
Harter, D. E., M. S. Krishnan, and S. A. Slaughter. 2000. Effects of process maturity on quality, cycle time, and effort in software product development. Management Science 46: 451-466.
Heinze, A. 2002. Software quality: The major challenge in the IT industry. Available at www.itdf.com/press/research/students/halexsej.pdf (last referred on July 4, 2004).
Huarng, F. 1998. Integrating ISO 9000 with TQM spirits: A survey. Industrial Management and Data Systems 8: 373-379.
lssac, G., C. Rajendran, and R. N. Anantharaman. 2004. A conceptual framework for total quality management in software organizations. Total Quality Management & Business Excellence: 307-343.
Jalote, P. 2000. CMM in practice. Reading, Mass.: Addison-Wesley.
Li, E. Y., H. G. Chen, and T. S. Lee. 2002. Software process management of companies in Taiwan: A comparative study. Total Quality Management 13: 701-713.
Niozi, M., D. Wilson, and D. Zowghi. 2003. A maturity model for the implementation of software process improvement: An empirical study. The Journal of Systems and Software. Available at: www.cse.dmu.ac.uk/~ieb/A%20maturilyimprovment.pdf (last referred on Nov. 20, 2007).
Parzinger, M. J., and R. Nath. 2000. A study of the relationships between total quality management implementation factors and software quality. Total Quality Management 11: 353-371.
Pun, K. F. 2001. Cultural influences on total quality management adoption in Chinese enterprises: An empirical study. Total Quality Management 12: 323-342.
Samson, D., and M. Terziovski. 1999. The relationship between total quality management practices and operational performance. Journal of Operations Management 17: 393409.
Sureshchandar, G. S., C. Rajendran, and R. N. Anantharaman. 2001. A conceptual model for TQM in service organizations. Total Quality Management 12: 343-363.
Sureshchandar, G. S., C. Rajendran, and R. N. Anantharaman. 2003. The influence of total quality service age on quality and operational performance. TQM & Business Excellence 14: 1033-1052.
Terziovski, M., and D. Samson. 1999. The link between total quality management practice and organizational performance. International Journal of Quality and Reliability Management 16:226-237.
Yang, H. Y. 2001. Software quality management and ISO 9000 implementation. Industrial Management and Data Systems 101: 329-338.
P. PADAAA, LS. GANESH, AND CHANDRASEKHARAN RAJENDRAN
INDIAN INSTITUTE OF TECHNOLOGY MADRAS
BIOGRAPHIES
P. Padma is a doctoral student in the Department of Management Studies, Indian Institute of Technology Madras, Chennai, India. She holds a bachelor of engineering degree in civil engineering from Madras University, and a master of science (by research) degree in industrial management from UT Madras. She is currently doing research for her thesis to be submitted for the award of doctoral degree in Indian Institute of Technology Madras. Her research areas are total quality management, service quality, and customer satisfaction. She has published articles in International Journal of Production Research and Benchmarking: An International Journal.
L S. Ganesh is a professor of management in the Department of Management Studies, Indian Institute of Technology Madras. His research interests are in project management, knowledge management, technology management, and decision making. He teaches courses on data analysis for management, systems thinking, and project management. He has many publications in international journals such as International Journal of Production Research, European Journal of Operation Research, and International Journal of Production Economics.
Chandrasekharan Rajendran is a professor of operations management in the Department of Management Studies, Indian Institute of Technology Madras. His research interests are in total quality management (TQM), scheduling, and simulation. He has published several articles in international journals, including Quality Management Journal, Software Quality Professional, Total Quality Management and Business Excellence, International Journal of Production Research, International Journal of Service Industry Management, International Journal of Bank Marketing, and Journal of Services Marketing. He serves as referee for many journals. He is a recipient of the Alexander van Humboldt Fellowship of Germany. Rajendran can be reached by e-mail at [email protected].
APPENDIX
Section A
Level 1
* Extent to which the processes are constantly changed as the work progresses.
* Presence of reaction-driven systems rather than effectively planned systems.
* Extent to which the performance of software processes depends on the capabilities of individuals and varies with their innate skills, knowledge, and motivations.
Level 2
* Extent to which realistic project commitments are based on the performance of previous projects.
* Extent to which the software project managers track software costs, schedule, and functionality. Level of adherence to commitments that are made after proper analysis of projects.
* Extent to which the organization defines software standards and ensures that the standards are faithfully followed.
* Level of presence of common understanding between customer requirements and project specifications.
* Establishment and maintenance of integrity of products/processes of a software project throughout the software life cycle.
* Establishment of reasonable plans for performing software engineering and managing software projects.
* Achievement of adequate visibility into actual progress so that management can take effective actions when the software project performance deviates from planned schedules.
* Extent to which software subcontractors are properly elected and managed effectively.
* Provision of appropriate visibility into processes of software project and of the products being built.
Level 3
* Degree to which the standard processes for developing and maintaining software across the organization are documented and integrated.
* Establishment of responsibility for software process activities that improve the overall software process capability.
* Development and maintenance of a usable set of software process assets to improve process performance and provide a basis for cumulative, long-term benefits.
* Extent to which the software project identifies the needed skills to the personnel and provides the necessary training when the project needs are unique.
* Integration of software engineering and management activities into coherent, well-defined software processes.
* Magnitude to which a defined software process is tailored from the organization's standard software processes.
* Performance of a well-defined engineering process that integrates all the software engineering activities to produce correct and consistent software products.
* Extent to which the technical activities (light requirements, analysis, design, code, and test) are described.
* Level to which interactions of software engineering groups with other groups are coordinated and controlled to satisfy customer's needs effectively and efficiently.
* Extent to which the software engineering products that undergo peer reviews are identified.
* Provision of adequate resources and funding to perform peer reviews on software products.
* Extent to which peer review leaders and reviewers receive required training.
* Extent to which the process includes readiness criteria, inputs, standards and procedures for performing the work, verification mechanisms, outputs, and completion of criteria.
* Degree of visibility of tasks in the projects to enable rapid and accurate status updates.
Level 4
* Intensity to which there is a quantitative basis for making decisions.
* Extent to which emphasis is laid on identification of the costs of variations within a measurably stable process, and correcting the causes of variation.
* Degree to which the process performance represents the actual results following the software process.
* Extent of development of the quantitative understanding of quality of project's software products.
* Application of a comprehensive measurement program to the software work products.
Level 5
* Degree to which the rework is primarily due to random causes/variations.
* Extent to which new and improved ways of building software are continually tried in a controlled manner to improve productivity and quality.
* Intensity to which the impact and effectiveness of change in the software processes are estimated and tracked.
* Extent of identification of beneficial new technology and transferring them into the organization in an orderly manner.
* Ability to focus on performing innovation efficiently in a dynamic environment.
* Presence of continuous improvement of software processes to increase productivity, decrease cycle time, and improve software quality.
* Extent to which the data are available and used to perform cost-benefit analysis of new technologies, and to propose changes to the organization's software processes.
* Extent of identification of weaknesses and strengthening the processes proactively, with the goal of preventing defects.
Section B
* Customer satisfaction
* Employee morale
* Profitability
* Overall productivity
* Reduction in quality costs
* Overall financial performance
* Overall operational performance
All the items in Section A and Section B are measured using two seven-point scales.
Copyright American Society for Quality 2008