Design Science Research: The Case of the IT Capability Maturity Framework (IT CMF)

Abstract

Design science (DS) is a problem solving paradigm that involves building and evaluating innovative artifacts in a rigorous manner to solve complex, real world problems, make research contributions that extend the boundaries of what is already known, and communicate the results to appropriate audiences. This paper presents a DS research project in the IS field that aims to improve organisational ability in managing and optimizing value realised from IT investments through increasing maturity in critical areas. This research involves development of an IT Capability Maturity Framework (IT CMF). This paper discusses its development in terms of key DS principles and presents reflections on the challenges and value associated with adopting a DS approach. The paper adds to the growing body of DS literature in the IS field, and enables other researchers and practitioners to judge the rigor with which the IT CMF artifact was created and evaluated, and its utility in practical application.

Headnote

Abstract: Design science (DS) is a problem solving paradigm that involves building and evaluating innovative artifacts in a rigorous manner to solve complex, real world problems, make research contributions that extend the boundaries of what is already known, and communicate the results to appropriate audiences. The importance of this paradigm in the Information Systems (IS) field has been recognised since the early 1990's with the publication of seminal articles by for example Nunamaker et al (1991), Walls et al (1992) and March and Smith (1995). However, over the past 15 years, DS research in IS has been sparse. In more recent times this has begun to change, with an increasing number of research contributions considering DS research. DS research in IS is important as the dominant behavioural science paradigm is not sufficient for addressing the types of problems that call for human creativity and innovative and novel solutions. One widely debated problem in the IS field that calls for such novel solutions centres on how organisations manage, deliver and optimise value from their IT investments. This paper presents a DS research project in the IS field that aims to improve organisational ability in managing and optimizing value realised from IT investments through increasing maturity in critical areas. This research involves development of an IT Capability Maturity Framework (IT CMF). The IT CMF project is centered at the Innovation Value Institute (IVI) at the National University of Ireland Maynooth (NUIM). The IVI is a joint venture between NUIM, Intel and the Boston Consulting Group and seeks to drive innovation in the management and use of IT in order to optimise business value. The IT CMF represents an emerging blueprint of key IT capability processes, and at a high level consists of four integrated IT management strategies or macro processes: managing IT like a business, managing the IT budget, managing the IT capability, and managing IT for business value. The IT CMF represents a blueprint for incrementally improving these four macro processes across five maturity levels: initial, basic, intermediate, advanced, and optimized. These four macro processes are further broken into 32 critical processes (CPs), which are the key activities that an IT organisation needs to manage in order to deliver IT solutions and measure the business value generated. The content development and review for the IT CMF is performed by the IVI development community, which comprises academic researchers, industry based practitioner-researchers and consultants based in over 55 global companies. This paper discusses its development in terms of key DS principles and presents reflections on the challenges and value associated with adopting a DS approach. The paper adds to the growing body of DS literature in the IS field, and enables other researchers and practitioners to judge the rigor with which the IT CMF artifact was created and evaluated, and its utility in practical application.

Keywords: design science, IT CMF, IS, case study, maturity models

1. Introduction

"What is design? Its where you stand with a foot in two worlds - the world of technology and the world of people and human purposes - and you try to bring the two together" (Kapor (1990) in Hevner and Chatterjee, 2010).

Design Science research is centered on building and evaluating artifacts in order to solve organisational problems. Much has been written about the research paradigm in other disciplines - its roots lie in engineering and the "sciences of the artificial" (Simon, 1996). Since the early 1990's, DS has been recognised as important in the Information Systems (IS) field in increasing an IT artifact's utility and effectiveness for solving complex business problems (Hevner and Chatterjee, 2010; Peffers et al, 2007). Over the past 15 years, IS DS research has been at best sporadic (Peffers et al, 2007; Walls et al, 2004) and publication in IS journals remains problematic (March and Storey, 2008). Despite this, Hevner and Chatterjee (2010) state that the IS field has witnessed a "flurry of recent activity" on the use of DS research.

This paper adds to the body of DS research in the IS field. It aims to establish the value of using a DS approach in the development of an IT Capability Maturity Framework (IT CMF) that seeks to help organisations to better manage and deliver value from their IT investments. The IT CMF focuses on four integrated strategies and 32 associated critical processes against which the IT organisation's level of maturity can be assessed according to five levels. The project has followed the DS research paradigm; this paper discusses its development in terms of key DS principles and concludes with a discussion of how this research approach has been of benefit.

The structure of this paper is as follows: Section two discusses the need for DS research in IS to complement the currently predominant behavioral science paradigm, so that the type of organisational problems that demand innovative and creative solutions can be addressed. Section Three summarises key principles drawn from seminal DS articles that underpin DS research projects. Section four provides an overview of the IT CMF project and discusses the DS approach to its content development in terms of Hevner's (2007) three DS cycles. Section five discusses the challenges and benefits associated with DS and draws conclusions to the research.

2. The need for design science research in IS

Both behavioural science and design science research paradigms are foundational to the IS discipline, which is positioned "at the confluence of people, organisations and technology" (Hevner et al, 2004). However, the prevalent research paradigm in the IS field to date has been behavioral science research (Hevner and Chatterjee, 2010). The objective of the behavioral science paradigm, which has its roots in natural science research methods, is problem understanding through developing and verifying theories on human and/or organisational phenomena that explain what happened, why it happened, and perhaps what will happen in a given context (March and Smith, 1995; March and Storey, 2008; Pries-Heje and Baskerville, 2008). March and Storey (2008) suggest that a typical question in this stream of IS research is "Why do investments in IT artifacts often not result in an increase in firm's value?" Its two key activities are regarded as discovery (i.e. generating new scientific claims such as theories or laws) and justification (i.e. testing such claims for validity) (March and Smith, 1995); the research output in such studies is often explanatory in nature (Peffers et al, 2007). March and Smith (1995) provide the following explanation:

"Natural scientists develop sets of concepts, or specialized language, with which to characterize phenomena. These are used in higher order constructions - laws, models, and theories - that make claims about the nature of reality. Theories - deep, principled explanations of phenomena - are the crowning achievements of natural science research. Products of natural science research are evaluated against norms of truth, or explanatory power. Claims must be consistent with observed facts, the ability to predict future observations being a mark of explanatory success. Progress is achieved as new theories provide deeper, more encompassing, and more accurate explanations".

Behavioural science often involves the development of a hypothesis, which is either proved or disproved with the collection and analysis of data by the researcher. Resulting theories provide insights pertaining to the interactions among people, organisations and technology that need to be managed. While this research paradigm is appropriate for studying existing and emerging organisational phenomena, there is a danger of over emphasising contextual theories at the expense of failing to anticipate new technological capabilities. This may result in behavioural science theories referring to out-dated or ineffective technologies. Further, the behavioural science paradigm is not sufficient for addressing the types of problems that call for human creativity and innovative and novel solutions (Hevner and Chatterjee, 2010; Peffers et al, 2007), for example, "What IT artifacts will increase firm value?" (March and Storey, 2008). In other words "science, the process of understanding "what is," may be insufficient for design, the process of understanding "what can be."" (Hevner and Chatterjee, 2010). These types of problems that require innovative solutions are regarded by Pries-Heje and Baskerville (2008), as ill-structured or "wicked problems", where requirements may be unstable, there may be complex interactions between problem subcomponents, and human cognitive and social abilities may be important in developing solutions (Hevner et al, 2004). Addressing these types of problems is the remit of DS research (March and Smith, 1995; March and Storey, 2008) and many such problems exist in the IS field.

While the behavioural science paradigm seeks to identify what is "true", the DS paradigm aims to create what is effective. DS is a problem solving paradigm that involves building and evaluating innovative artifacts in a rigorous manner to solve complex, real world problems, make research contributions that extend the boundaries of what is already known, and communicate the results to appropriate audiences (Adomavicius et al, 2008; Gregor and Jones, 2007; Hevner et al, 2004; March and Smith, 1995; March and Storey, 2008; Pries-Heje and Baskerville, 2008; Purao, 2002; Vaishnavi and Kuechler, 2004/5; Venable, 2006). Knowledge and understanding of the problem domain is achieved through artifact construction (Hevner et al, 2004), which must have novelty and utility in the application environment (Hevner and Chatterjee, 2010; March and Storey, 2008; Simon, 1996). Analysis of the utility and performance of the developed artifacts provide improved understanding and identification of further improvements that enable the business problem/need to be addressed more effectively. According to Peffer et al (2007) the "design and the proof of its usefulness is the central component". DS research in the IS field is not limited to IT artifacts in the form of computer based systems. Artifacts or solution technologies may include IS development methods, tools and techniques, IS security and risk management practices, and IS planning and management methods (Venable, 2006).

It should be noted that because technology and behavior are inseparable in IS research (Hevner et al, 2004), both behavioral science and DS research paradigms are increasingly recognised as "complementary partners" (Hevner et al, 2004) or "equal companions" (March and Storey, 2008) in addressing important IS problems. Behavioural science theories are influenced by IS design decisions; similarly the outputs from DS are influenced by behavioural theories. In fact DS artifacts "creation relies on existing kernel theories that are applied, tested, modified, and extended through the experience, creativity, intuition, and problem solving capabilities of the researcher" (Hevner et al, 2004). In other words "truth informs design and utility informs theory". These researchers argue for the need to engage in the complementary research cycle between behavioural science and design science in order to address important IS problems, as both are needed to ensure IS research is effective and relevant. They state that "IS research must be both proactive and reactive with respect to technology. It needs a complete research cycle where design science creates artifacts for specific information problems based on relevant behavioural science theory and behavioural science anticipates and engages the created technology artifacts".

3. Design science principles

DS research is underpinned by a number of principles, which are summarised in this section.

Principle One - Design Artifact: IS design science research involves developing useful artifacts that address relevant business/IT problems (March and Smith, 1995; Hevner et al, 2004; Hevner, 2007, Iivari, 2007; Baskerville, 2008; Peffers et al, 2007). These IT artifacts include constructs (vocabulary, symbols, or conceptualizations used to describe problems or solution components); models (abstractions and representations of the problem and its solution space); methods (algorithms, practices and processes providing guidelines in performing tasks or searching the solution space); and instantiations (implemented or prototype systems demonstrating utility of the IT artifact in addressing specific tasks) (March and Smith, 1995).

Principle Two - Design Problem Relevance: Benbasat and Zmud (1999) in the MIS Quarterly paper "Empirical Research in Information Systems: The Practice of Relevance" highlight a strong need for more relevance in IS research. They argue that to date an over emphasis has been placed on rigor over relevance, often resulting in research outputs that are not prescribed in a manner that can be easily used by practitioners to address a problem or opportunity. DS research is problem driven (Baskerville, 2008). As stated by Hevner et al (2004), "research must address the problems faced and the opportunities afforded by the intersection of people, organisations, and information technology". The DS Relevance Cycle proposed by Hevner (2007) connects the research project's contextual environment with DS activities. This contextual environment sets out the problem space which includes the phenomena of interest i.e. the problems, opportunities, goals and tasks defining business needs of the organisation as perceived by individuals and their positioning relative to existing technological architecture and capabilities. Together these set out the problem or business need that needs to be addressed. Problems or opportunities identified in the application domain (i.e. the research requirements) initiate DS research. The problem addressed, i.e. the difference between the goal state and current state of a system, needs to be relevant to the community of practitioners involved in planning, designing, managing, implementing, operating and evaluating IS and technologies. Field testing of the built artifact in the application domain based on predefined acceptance criteria determines if further iterations of this relevance cycle are needed.

Principle Three - Design Cycle: DS activities iterate between building design artifacts and processes and evaluating them in order to provide feedback for further refinement (March and Smith, 1995; Hevner et al, 2004; Hevner, 2007). These design cycle activities are based on both relevance and rigor, focused on addressing specific application domain requirements, while drawing on existing theoretical foundations and methodologies in the knowledge base (Hevner and Chatterjee, 2010).

Principle Four - Design Research Rigor: Rigor refers to "the correct use of methods and analyses appropriate to the tasks at hand" (Benbasat and Zmud, 1999); in DS terms it is determined by the researcher's ability to select suitable techniques to build and evaluate the artifact (Hevner et al, 2004). The DS Rigor Cycle (Hevner, 2007) links DS build and evaluate activities with the knowledge base that informs the research project. This knowledge base includes existing foundational/kernal theories, frameworks, artifacts, processes, methodologies, experiences and expertise within the application domain; and the DS research is grounded in these existing ideas. A challenge of DS is the fact that the existing knowledge base is often insufficient as some theories in the knowledge base may be undeveloped or incomplete. Further, there is an insufficient set of tools, constructs, models and methods to represent the business/technology environment accurately, with abstract representations often regarded as having poor relationships with the real world environment. Finding representational techniques that balance rigor and relevance is difficult (Hevner et al, 2004). Inspiration for design activities may also come from creative insights, gut instinct, or "imaginative jumps to future possibilities" (Purao, 2002), and the very process of building the IT artifact may enhance understanding and development of those theories (Hevner and Chatterjee, 2010). Because no knowledge base is complete for every situation, the design process contributes important knowledge to the incomplete theories that motivated the design project in the first place.

Principle Five - Design Artifact Evaluation: "The essence of Information Systems as design science lies in the scientific evaluation of artifacts" (Iivari, 2007). Rigorous evaluation methods are required to demonstrate the design artifact's utility, quality and efficacy. The evaluation process helps researchers to understand the nuances in their design and contribute to the body of knowledge to facilitate learning by future researchers (Hevner and Chatterjee, 2010). Applying rigorous evaluation represents one of the key DS challenges (Hevner et al, 2004; Iivari and Venable, 2009; March and Vogus, 2010). Evaluation episodes include artifact evaluation to refine its design during the build-evaluate cycle and field testing of the artifact in its application environment (Hevner, 2007). Similarly, Pries-Heje et al (2008) suggest two distinct evaluation episodes: design-evaluate and construct-evaluate. Two key steps involved in artifact evaluation involve selecting the evaluation technique and evaluation metrics (Hevner and Chatterjee, 2010). Metrics are used in comparing the performance of constructs, models, methods and instantiations for specific tasks. For example, constructs may be evaluated in terms of completeness, simplicity, ease of use; models may be examined with respect to their fidelity with real world phenomena, level of detail, robustness, and practical utility; methods may be evaluated in terms of operationality, efficiency, generality and ease of use, while instantiations evaluation considers artifact efficiency and effectiveness and its impact on the application domain's end users (March and Smith, 1995). The evaluation method selected from the knowledge base must be appropriate for the artifact in question. Evaluation approaches may include observational methods (case studies, field studies); analytical methods (static analysis, dynamic analysis); experimental methods (controlled experiments, simulations); testing (functional, structural); or descriptive methods (informed arguments, scenarios) (Hevner et al, 2004). These mirror closely with the naturalistic and artificial evaluation approaches proposed by Pries-Heje et al (2008).

Principle Six - Design Research Contributions: Contributions of DS research include an artifact that adds to the existing knowledge base or uses existing knowledge in innovative ways (Hevner et al, 2004; March and Smith, 1995); design construction knowledge extending/improving existing foundations in the knowledge base; and/or design evaluation knowledge enhancing existing methodologies (Hevner et al, 2004). Experience gained from the iterative design and artifact evaluation are also valuable contributions.

4. The case: Development of the IT CMF using DS principles

4.1 IT CMF introduced

The IT CMF project is centred at the Innovation Value Institute (IVI), at the National University of Ireland Maynooth. IVI is a research centre seeking to drive innovation in the management and use of IT. The IT CMF (Figure 1) is an innovative and systematic framework, enabling CIO's/CEO's to understand and improve their organisation's maturity and enable optimal business value realisation from IT investments (Curley, 2004; 2007). The framework represents an emerging blueprint of key IT capability processes and acts as an assessment tool and a management system with improvement maps that help organisations to continually improve their IT capability over five levels of maturity - initial, basic, intermediate, advanced, and optimised (Table 1). The meta-elements of the IT-CMF can be depicted in three interlinked layers, namely strategy, macro and micro layers.

[1] The strategy layer underpins the primary elements of the IT-CMF that support an approach to strategic thinking comprising business context driven by the organisation's vision of its future; business strategy; IT capability; business operations; and, business value (Curley, 2004).

[2] The Macro layer consists of both the content and context of application of the IT-CMF. The content segments the activities of an organisation's IT function into four macro-processes (MPs) namely: Managing IT like a business, Managing the IT budget, Managing the IT capability and Managing IT for business value. These four integrated IT management strategies underpin value oriented IT management.

[3] The Micro-layer comprises 32 critical processes (CPs) assigned to the four individual macro processes. These represent key activities of the IT organisation in delivering IT solutions and optimising the associated business value generated. Each CP encompasses a number of categories and capability building blocks (CBBs), which reflect the CPs content and assumptions associated with each of the five maturity levels (Curley 2004). Understanding an organisation's current and desired maturity levels helps set improvement initiatives that drive value delivery. Improving maturity across these CPs reflects organisational progress over time.

4.2 Overview of IT CMF development

The content development and review process for the IT CMF is performed by the IVI consortium. A work group is formed for each of the 32 CPs, which consists of a mix of Subject Matters Experts (SMEs) and Key Opinion Leaders (KOLs) on a specific topic, including academic researchers, industry-based practitioners, and consultants. The work groups are led by an individual, who acts as a facilitator for the group in achieving its goals and objectives. Development work involves creating detailed content for each of the CPs based on a standard template or blueprint. Such content includes for example:

* An industry standard process definition,

* In scope and out of scope aspects,

* Definition of categories and capability building blocks (CBBs) for each CP,

* Definition of relations between CPs,

* Differentiation with industry IT management frameworks,

* A five level maturity curve framework representing the maturity underpinning the scope of the categories and CBBs defined for each CP,

* A detailed assessment tool comprising maturity level questions for each CBB,

* A set of practices, outcomes and metrics associated with each CBB at each level of maturity,

* A collection of other reference documents, such as quick scan cards, illustrative examples, marketing booklets, interview guides etc.

Work group output evolves through a series of four stages (Table 2) and is reviewed at the end of each stage by a technical committee (TC). This TC may approve progression to the next stage or request further development work before progression status is granted. As development work progresses through the various stages, more in-depth content is required and the CP material is subject to more rigorous reviews and validation processes.

4.3 Adopting a design science approach

As outlined in section 3, DS is centered on developing useful artifacts, which take the forms of constructs, models, methods and instantiations [Principle One - Design Artifact]. In DS research, maturity models are located between models and methods in the sense that a) they offer state descriptions on the current maturity level assessment and b) guidelines with respect to how organisations can achieve higher maturity. The first aspect can be considered a model perspective; the second outlines guidelines in the form of method components (Donnellan and Helfert, 2010). The DS process followed in developing the IT CMF artifact over the four stages of content development can be examined in terms of the three DS research cycles (Figure 2) proposed by Hevner (2007).

The development of the IT CMF is underpinned by both relevance and rigor. As outlined in Section 3, DS research is problem oriented [Principle Two - Design Problem Relevance]. The DS relevance cycle acts as the link between the research projects context and the initiation of DS activities, through identification of a relevant Business/IT problem. The relevance of the IT CMF project is driven by the problems experienced by organisations in optimizing how they currently manage and measure the business value of their IT investments. IT value has been a subject of debate for many years. Linked to the productivity paradox literature (e.g. Brynjolfsson, 1993) that emerged in the 1980's, Robert Solow (1987: 36) is well cited for stating "you can see computers everywhere, except in the productivity statistics". The concept of value in use, which refers to the net benefit stream derived through IT usage, is quite complex to quantify and realise and a significant component of IT value comes from the ability to leverage associated processes and work practices to differentiate from competitors (Markus, 2004; Ranganathan and Brown, 2006). At present, there are approximately 30 IT management frameworks, for example Information Technology Infrastructure Library (ITIL), Control Objectives for Information and related Technology (COBIT), Capability Maturity Model Integration (CMMI), IT Balanced Scorecard, PRINCE2, ValIT and so on. While offering best practices and guidance in several areas, these IT management frameworks do not address the full scope of the IT CMF framework and have not resolved the issue of enabling organisations to optimise the value delivered from their IT investments. Hence, the problem of managing and delivering business value from IT remains of significant relevance as annual IT spend (currently estimated at £1.6 trillion (Gartner, 2010)) continues to grow, as the pace of technology intensifies and as organisations become more and more reliant on IT. The issue of optimizing IT enabled business value is therefore a problem requiring an innovative solution and is suited to a DS research approach.

The DS rigor cycle [Principle Four - Design Research Rigor] acts as the link between the knowledge base that informs the project and the design activities. The IT CMF development is grounded in existing artifacts, methodologies, foundational theories and expertise. The research has theoretical foundations in the field of IT management and draws from an extensive base of industry and academic literature. The artifact leverages existing IT standards and frameworks, which have widespread application by organisations in their IT management.

Grounded in both relevance and rigor, the design cycle [Principle Three - Design Cycle] of the IT CMF focuses on iterative build and evaluate activities by the CP work groups across the four stages of its development. As illustrated in Figure 2, each stage of development is a refinement of the previous stage, adding greater depth of detail. Within the design cycle, the build process is evolved and refined through feedback provided from evaluation activities [Principle Five - Design Artifact Evaluation] using methodologies existing in the knowledge base. Evaluation activities for the IT CMF include the use of the informed argument method by the workgroup members, technical committee stage gate reviews; and field testing of the artifact with organisations.

*The use of the informed argument evaluation method (Hevner et al, 2004), where information from the knowledge base is used to build convincing arguments for the artifacts utility, is appropriate for the IT CMF. During content development by the workgroup, the IT CMF is compared with several IT management frameworks; for each CP the degree of fit/match between it's CBBs and relevant IT management frameworks and industry standards is visually depicted. An examination of this industry framework scan highlights the gaps existing in current IT management standards, many of which are addressed by the IT CMF. This model also differs from existing IT management frameworks in that it adopts a holistic organisational approach and provides guidelines to contextualize maturity models to take into consideration organisational size, sector, structure, IS decision making structure, communication structure, and task structure. Informed arguments of this nature are one means of iteratively evolving the IT CMF design.

*Further, within the design cycle, the evaluation and stage gate reviews by a technical committee serve as important feedback on learnings from the review process and refinements required by the workgroup before the CP can advance to the following stage. Such feedback incorporates the fresh insights of IVI consortium members who have not been closely involved in the CP's development.

*As outlined in Section 3, design artifact evaluation is not limited to the design cycle. Field testing of the artifact in its application environment is necessary to demonstrate its utility, quality and efficacy and the feasibility of the approach to solving the problem. This again links together the DS activities and the application environment via the relevance cycle. From an observational evaluation approach (Hevner et al, 2004) perspective, instantiations of the IT CMF are evaluated in contextually diverse organisations using the case study method. As outlined in Table 2, in order for a critical process to progress from development stage 3 to stage 4, the CP material is formally assessed in at least three organisations. This typically involves conducting CP pilot assessments within organisations; the assessors who lead this activity capture feedback from the organisations via a standard CP Pilot assessment report template. This report captures insights on the CP's ability to assess the organisation's process maturity; the scope of the CP definition; capability building blocks, maturity levels, POMs, and questionnaire; and any other insights on how the CP material could be improved. Such field testing based on predefined acceptance criteria determines if further iterations of the relevance cycle are needed and provides valuable feedback on the utility of the maturity level assessment approach, its comprehensiveness, its understandability, and its value in assessing current and setting target maturity levels. Incorporating such feedback into the relevant CP material becomes a further important build activity within the design cycle for the IVI workgroups.

The output of a DS research activity is a contribution to the knowledge base informing the project [Principle Six - Design Research Contributions]; thereby linking the design activities and the knowledge base via the rigor cycle. Such contributions may be in the form of an artefact, an extension to an existing foundational theory and/or new design evaluation knowledge. Novel contributions from this project included the IT CMF artifact providing a detailed framework that helps drive innovation and change in how organisations manage and use their IT investments to optimise business value. Application of this artifact in the business environment helps organisations to determine their current CP maturity levels, and identify improvement initiative priorities to increase their IT capabilities efficiency and effectiveness in delivering value. This helps address a perennial business IT problem. Contributions further extend existing foundational theories by addressing the gaps in existing IT management frameworks and standards and recognising that the IT CMF can defer to other frameworks in the knowledge base for additional actionable practices.

5. Discussion and conclusions

DS has provided a useful methodological approach to development of the IT CMF. While the prevalent behavioural science research paradigm facilitates explanations and development of theories on the problem of realising business value from IT investments, DS enables the development of novel, innovative ways to address the problem. The value of the DS approach in this project lies in the insights gained through the iterative build-evaluate activities. The build-evaluate cycle enhanced understanding of the business/IT problem and the complex interactions of problem subcomponents. This level of understanding could only be gained through the specific act of building and testing an artifact iteratively. The approach resulted in improved understanding by the IVI research team regarding the utility of the developed CP material in assessing organisational maturity in a specific process. It offered inherent flexibility to change and evolve the artifact until it was useful and effective in addressing the problem of how organisations can optimise value delivery from IT investments.

The iterative design process, incorporating input from WG participants, TC members, and case study organisations, provides diverse perspectives that enhance the IT CMF artefact's quality and utility. Grounding the research in existing theoretical and methodological foundations and expertise ensures that the research avoids the risk of "reinventing the wheel", builds on an already established knowledge base, and offers a contribution that addresses the deficiencies of existing IT management frameworks. This in-depth approach to content development is essential in demonstrating the rigor with which the IT CMF was created, and ultimately in gaining buy-in for the framework as a novel and holistic approach to IT management. Its utility is evident in the following exemplar results from applying IT CMF practices: a 25% improvement in IT capability for 10% reduction in IT spend at Intel; 96% reduction in set up working time for new servers at Axa-Tech; and an 8% saving in total operating budget for Technology Innovation and 20% saving of total budget for experiment execution at Merck (Grant, 2010).

It is important to note that the DS approach to development of the IT CMF is not without its difficulties. Within the design cycle, challenges exist in aligning the input and feedback of a diverse range of researchers and practitioners during the build-evaluate activities and as such the design process is reliant on the human cognitive and social abilities of the workgroup participants. An aligned understanding and definition of the problem relevance for each CP is required. Ensuring rigor in the content development throughout the various stages requires iterative refinement through ongoing literature search to incorporate up-to-date methods, frameworks, practices etc. Validation of material within and between stages, results in a longer development time for some CPs, with the need in some cases to revise material content and conduct further research prior to the technical committee granting approval for material to progress to the next stage. This longer development time results in the CPs of the IT CMF reflecting different stages of development at a specific point in time. Despite these challenges the DS approach ultimately results in a higher quality artifact.

Overall, DS research provides a robust approach towards addressing the type of ill-structured business/IT problems faced in the IT CMF research project. It complements behavioural science research oriented towards problem understanding and providing explanations, by developing in a rigorous manner novel solutions that extend the boundaries of organisational capabilities. These novel solutions or artifacts serve as the basis of future behavioural science theories, which will inform the underlying knowledge base for future DS activities. The complementarity between these two paradigms enables relevant IS problems to be more effectively addressed. The more recent surge of interest in DS as a research paradigm in IS is of significance importance in advancing the IS field in innovative and novel ways. This paper calls for the more widespread publication of DS oriented research in mainstream IS and research methods journals.

References

References

Adomavicius, G., Bockstedt, J.C., Gupta, A. and Kauffman, R.J. (2008). Making sense of technology trends in the information technology landscape: a design science approach. MIS Quarterly, 32, (4), 779-809.

Baskerville, R. (2008). What design science is not. European Journal of Information Systems, 17(5), 441-443.

Benbasat, I. and Zmud, R.W. (1999). Empirical research in information Systems: the practice of relevance. MIS Quarterly, 23, (1), 3-16.

Brynjolfsson, E. (1993). The productivity paradox of Information Technology. Communications of the ACM, 35, 66-77.

Curley, M. (2004). Managing information Technology for business value - practical strategies for IT and business managers. Intel Press.

Curley, M. (2007) Introducing an IT Capability Maturity Framework, International Conference on Enterprise Information Systems.

Donnellan, B. and Helfert, M. (2010). The IT CMF: A Practical Application of Design Science. Global Perspectives on Design Science Research - Lecture Notes in Computer Science. 6105, 550-553.

Grant, I. (2010). Early adopters report big savings from the IT-CMF. Computer Weekly. Available from http://www.computerweekly.com/Articles/2010/02/25/240420/early-adopters-report-big-savings-from-it-cmf.htm. 25 February.

Gregor, S (2006). The nature of theory in Information Systems. MIS Quarterly. 30, (3), 611-642.

Gregor, S. and Jones, D. (2007). The anatomy of a design theory. Journal of the Association of Information Systems, 8, (5), 312-335.

Hevner, A. (2007) A three-cycle view of design science research, Scandinavian Journal of Information Systems. 19 (2), 87-92.

Hevner, A., March, S. and Park, J. (2004). Design Science in Information Systems research. MIS Quarterly. 28, (1), 75-105.

Hevner, A., Chatterjee, S. (2010). Design Science Research in Information Systems, Springer Science and Business Media.

Iivari, J. (2007). A paradigmatic analysis of Information Systems as a design science. Scandinavian Journal of Information Systems. 19, (2). 39-64.

Iivari, J. and Venable, J. (2009). Action research and design science research - seemingly similar but decisively dissimilar. In Proceedings of the 17th European Conference on Information Systems. Verona, June 8th - 10th.

March, S. and Smith, G. (1995) Design and natural science research on information technology, Decision Support Systems 15, (4), 251-266.

March, S.T. and Storey, V.C. (2008). Design Science in the Information Systems discipline: an introduction to the special issue on design science research, MIS Quarterly, 32, (4), 725-730.

March, S.T. and Vogus, T.J. (2010). Design science in the management disciplines. In Hevner, A. and Chatterjee, S. Design Science Research in Information Systems, Springer Science and Business Media, 195-208.

Markus, M.L., Majchrzak, A. and Gasser, L. (2002). A design theory for systems that support emergent knowledge processes. MIS Quarterly, 26, (3), 179-212.

Markus, M.L. (2004). Technochange management: using IT to drive organisational change. Journal of Information Technology, 19, March, 4-20.

Nunamaker, J.F., Chen, M., and Purdin, T.D.M. (1991). "System Development in Information Systems Research," Journal of Management Information Systems (7:3), pp. 89-106.

Peffers, K., Tuunanen, T, Rothenberger, M.A. and Chatterjee, S. (2007). A Design Science research methodology for information systems research. Journal of Management Information Systems. 24, (3), 45-77.

Pries-Heje, J. and Baskerville, R. (2008). The Design Theory Nexus, MIS Quarterly, 32, (4), 731-755.

Pries-Heje, J., Baskerville, R., and Venable, J. (2008). Strategies for design science research evaluation. Available from Internet http://is2.lse.ac.uk/asp/aspecis/20080023.pdf.

Purao, S. (2002). "Design Research in the Technology of Information Systems: Truth or Dare." Georgia State University, Department of CIS Working Paper. Atlanta.

Ranganathan, C. and Brown, C.V. (2006). ERP investments and the market value of firms: toward an understanding of influential ERP project variables. Information Systems Research, 17, (2), 145-161.

Simon, H. (1996). The Sciences of the Artificial, Third Edition. Cambridge, MA, MIT Press.

Solow, R.M. (1987). We'd better watch out. New York Times Book Review, July 12th, pp 36.

Vaishnavi, V. and Kuechler, W. (2004/5). "Design Research in Information Systems" January 20, 2004, last updated August 16, 2009. URL: http://desrist.org/design-research-in-information-systems.

Venable, J.R. (2006). The role of theory and theorizing in design science research. In Proceedings of the First International Conference on Design Science Research in Information Systems. 24th-25th February, Claremont, CA.

Walls, J.G., Widmeyer, G., El Sawy, O. (1992). Building an Information System Design Theory for Vigilant EIS, Information Systems Research, 3, (1), 36-59.

Walls, J.,Widmeyer, G., El Sawy, O. (2004). "Assessing Information System Design Theory in Perspective: How Useful was our 1992 Initial Rendition." Journal of Information Technology Theory and Application. 6, (2): 43-58.