IRMAS - development of a risk management tool for collaborative multi-site, multi-partner new product development projects

Abstract

Purpose - The purpose of this paper is to develop comprehensive risk management tool, Intelligent Risk Mapping and Assessment System (IRMAS(TM)) with a contingency for multi-site, multi-partner concurrent engineering projects with the aim of achieving above-mentioned paradigms. Its unique knowledge warehouse enables the use of organisational knowledge, lessons learnt, captured as well as best practices to minimise risks in project management. Design/methodology/approach - IRMAS is designed to identify, prioritise, analyse and assist project managers to manage perceived sources of concurrent engineering risks. Several knowledge elicitation techniques were used to compile the knowledge used for the intelligent system developed. The core of the research is the reasoning methodology that not only supports the decision-making process of the user, but also aids the knowledge retrieving, storing, sharing and updating process of manufacturing organisations. Findings - A total of 589 risk items were identified for different project types, as well as information on 4,372 risk items and 136 lessons learnt were gathered. IRMAS is a proactive tool supporting project management activities. It is designed as a web-based portal compiled in Java facilitating effective and a common communication platform between project partners. Research limitations/implications - Identification of risks during the complete product design, development and delivery process in a concurrent engineering environment is challenging. It covers the "product value stream" including partners, suppliers, research and development, design and manufacturing, marketing, purchasing, service and support personnel and customers. Within the context of concurrent engineering, the design style must be "Design WITH" approach where collaborative negotiation requires communication, consideration and collaboration. The full validation of IRMAS(TM) is successfully carried out in two large-scale new product development projects. It has already been decided to be deployed by a large international aerospace company and is successfully commercialized. Originality/value - The originality of the paper lies in its uniqueness in these areas: IRMAS provides a systematic engineering approach to risk management of concurrent product and process development based on risk management standards and Project Management Body of Knowledge, to leverage of success factors in manufacturing; concurrencies and relationships between several activities throughout product's life cycle are captured and mapped; the inheritance of risk between several phases are modelled and quantified; the wealth of knowledge stored in the knowledge repository and IRMAS's capability to reuse them for later elicitation in the system's knowledge base; and user-interactive, unique dynamic risk management software package which will be available in the commercial market. [PUBLICATION ABSTRACT]

Introduction

The management of risks in multi-site, multi-partner (MSMP) new product development concurrent engineering (CE) projects attract interest from both academia and practitioners. In Project Management Body of Knowledge (PMBOK), risk management has been indicated as one of the eight main areas to consider ([33] PMI, 1996). Risk management is a process which covers the life cycle of product design, development and delivery (PD³ ). Project planning, organising, managing and controlling phases must cover (PD³ ) in depth where several risks are housed. Most authors emphasise that it is important to understand exactly what is meant by risk before it can be managed. There are numerous definitions of risk which have changed only little over the decade focusing on the likelihood of occurrence and the degree of impact of a negative event adversely affecting any activity ([33] PMI, 1996; [25] Larson and Kusiak, 1996; [17] Jaafary, 2001; [10] Conroy and Soltan, 1998; [12] Craig, 2003; [36] Smith and Merritt, 2002). The definition by [38] Wideman (1986) places risk in the context of project management. He defines project risk as "the chance of certain occurrences adversely affecting project objectives". In other words, the project risk is "an event, which should it occur, would have negative effect on the achievement of a project's objectives". This definition is adopted to identify risks in process in this paper.

Identification of risks are considered by many authors to be the most important element of the complete (PD³ ) process, as once it has been identified it is possible to take action to address it ([38] Wideman, 1986; [11] Cooper and Chapman, 1989; [5], [6] CCTA, 1993, 1994; [7] Chapman and Ward, 1997; [30] Perry and Hayes, 1986; [16] Hertz and Thomas, 1983). Thus, the success of the identification process to a very large degree is dependent on the "complete understanding of the new product design and development in a multi-partner, multi-location CE environment" as discussed in this paper.

The new technologies, computing and communication have become indispensable in every aspect of the design and manufacturing process, leading to structural changes in social and economical dimensions. Internet technology has led to e-manufacturing which boosted tremendously marketing and sales operations of organisations but not operational aspects of manufacturing. Through business to customer (B2C) and business to business (B2B), customers and businesses have become interconnected carrying out fast on-line transactions. The full scale involvement of manufacturers at operational levels have not been achieved yet due to a lack of complete understanding of the project life cycle. According to [37] Tseng et al. (2003) this is mainly because every aspect of engineering design and/or manufacturing capabilities have not been linked with customers and suppliers proactively throughout the (PD³ ) process and collaborating across boundaries. The significant impacts would be on three major areas, namely:

Speed of decision (the exchange of information including requirements, drawings, models, test results, etc. dramatically reduced time to market, cost of uncertainty and inventory in product design and development).

Expansion of scope (the web inter-connectivity integrated contributions to product design and development regardless of time, geographical distances, stakeholders, suppliers and customers anywhere around the world).

Degree of concurrency (people as well as machines can interact in parallel inside and outside of organisations, anywhere around the world).

Thus, to transform from designing products to designing the complete (PD³ ) process - is rewarding but challenging as well, introducing several risks to manufacturing projects.

This paper examines the new challenges in product design and development in CE projects in manufacturing industries which expose them to several risks. The Intelligent Risk Mapping and Assessment System (IRMAS(TM)) being developed will be presented briefly, covering the risk identification stage of the system in more detail. IRMAS(TM) aims to provide a systematic approach to identify, assess and mitigate potential risk items at all stages of the project life cycle, thus preventing problems and failures to occur. It is designed to be used as a decision support tool for decision-makers, such as project managers.

Background

The collaborative design process inherits several risks due to knowledge sharing, decision sharing, process sharing and resources sharing in CE projects. In this section of the paper, product design in CE environment and management of risks in such an environment will be discussed.

Concurrent engineering

CE is now widely understood in many industries as a common approach for developing new products in an integrated manner through the use of multi-functional teams; early involvement of all enterprise functions that contribute to a successful product; and overlapping of hitherto sequential activities ([37] Tseng et al. , 2003; [31] Prasad, 1996). This approach is intended to cause every party involved in new product development of the product life, from conception through disposal including quality, cost, schedule and user requirements. So it is not just about the integration of "engineering" functions. It covers the "product value stream" including suppliers, marketing, purchasing, service and support personnel and obviously the customers.

Nowadays, the possibility to reach anywhere and anyone immediately through internet is very economical. The instant transmission of information between every partner of a project as well as flexibility of manufacturing systems help manufacturers avoid several risks.

Designing the complete (PD³ ) process approaches customers as an important stakeholder and consider suppliers as partners. This is intended to extend manufacturing capability while focusing on core competencies of each. The paradigm then is to view design as teamwork and achieve collaborative effort through effective communication among geographically distributed partners. The dilemma then is information sharing, collaborative decision making, compatibility of processes and resource sharing, leading to enhanced effectiveness and efficiency of the product design and development on one hand, while introducing new risks on the other. For example, initial studies of merging two big truck companies in Sweden, Scania and Volvo showed that out of 150 important terms used in their product design, only five were common to both companies. Still those five did not have exactly the same meaning ([15] Haque, 2003).

A methodology to analyse a collaborative design process and management of product design conflicts was developed by [27] Lu and Cai (2000) and [28] Lutters et al. (2001). Within the context of new e-manufacturing, the design style must be changed from the "Design OF" in the past, through the "Design FOR" at present, to the "Design WITH" in the future ([37] Tseng et al. , 2003).

The key feature of "WITH" approach is that designers continuously and collaboratively negotiate their decisions with all other stakeholders around the globe across the internet. Collaborative negotiation requires communication, consideration and collaboration supports beyond those traditional design approaches, which relied on iterations.

[27] Lu and Cai (2000) proposed Engineering as Collaborative Negotiation (ECN) paradigm to address this problem of collaborative design in the new e-manufacturing era. ECN is an interactive co-construction process where decision interdependencies, mutual adjustments, conflict of priorities, misunderstandings, etc. are investigated. Under the ECN paradigm, the socio-technical framework was developed for collaborative design ([26] Lu and Cai, 1999), which takes the view that engineering design is a "technical" activity with a "human" purpose. Modern engineering design always involves multiple stakeholders with competing life-cycle concerns, and hence, is a complex socio-technical activity. In today's collaborative design environment, the number of project participants have been increased and the nature and means of collaboration has been changed with different participant backgrounds, interests, expertise, behaviours, cultural features, etc. This complexity is greatly multiplied in the e-manufacturing economy, because the world wide web facilitates coupling, promotes conflicts and communication problems among project teams with different educational, cultural and social backgrounds ([37] Tseng et al. , 2003).

CE projects require multiple parties to work collaboratively. Particularly with e-manufacturing, the number of participants have increased. For example, supplier collaboration early in the design reduces products lifecycle cost and extend company's ability beyond its traditional boundaries for improvements and to improve total cost of doing business together ([28] Lutters et al. , 2001). Hence, to increase the chances of success for manufacturing organisations, [27] Lu and Cai (2000) emphasised the importance of collaboration between project partners during engineering design especially in these three areas: task decomposition and representation, communication infrastructure and collaboration support. This collaboration will eventually lead to competitiveness of organisations, due to better knowledge utilisation and sharing with every project partner and incorporating the changing design style from the "Design OF" in the past, via the "Design FOR" at present, to the "Design WITH" in the future. In order to achieve successfully "Design WITH" in CE projects, knowledge management needs to be incorporated into the risk management practices of manufacturing organisations.

Risk management

Risk management is one of the eight main areas of the PMBOK ([33] PMI, 1996). Risk management is a life-cycle process which covers from the definition until customer service of the project.

Several approaches to risk management have been proposed. The Software Engineering Institute has identified five phases (identification, analysis, response development and control) ([37] Tseng et al. , 2003). Also Project Management Institute covered four phases of the risk management process (identification, quantification, response development and control) ([13] Durofee et al. , 1996). [22] Klein and Cork (1998) described a four phase process (identification, analysis, control and reporting). [3] Boehm (1991) suggested risk assessment (identification, analysis and prioritisation) and risk control (risk management planning, risk resolution and risk monitoring planning, tracking and corrective action). [8] Chapman (1998) proposed nine phases: (definition, strategic approach, identification of risks, information structuring, ownership, uncertainty estimation, magnitude of risks, response, monitoring and controlling).

[14] Fairley (1994) proposed seven steps (identification), assessment, mitigation, monitoring, contingency planning, managing the crisis and recovery from the crisis.

Risk management as per Australia and New Zealand Risk Management Standard (AS/NZ 4360) comprises of seven iterative processes; establishing the context, identification of risks, analysis of risks, evaluating risks, treating risks, communication and consultation and monitoring and review ([2] AS/NZ Standard 4360, 1999). Like above approaches, the Riskman methodology provided a framework for risk analysis and control ([4] Carter et al. , 2001).

Thus, risk management methodologies are generally composed of four stages: risk identification, assessment, treatment (mitigation) and monitoring ([10] Conroy and Soltan, 1998; [34] Raz and Michael, 2001).

Several risk types are encountered in concurrent product development process. These risks are also covered in the project management literature ([9] Chapman, 2001; [18] Kayis et al. , 2003; [24] Kusumo et al. , 2003; [39] Williams, 1995; [19] Kara et al. , 1999; [20] Kara et al. , 2001; [41] Zhou et al. , 2002). Risks can be of different categories such as; risks linked to the project (e.g. characteristics of the project, team skills, new product development or derivative product development, technical risks, communication risks, schedule risks, etc.), internal risks (e.g. customer, resource risks related to personnel, machinery, equipment, tools, organisational structure, stakeholders, leadership, organisational culture, location risks, etc.) and external risks (e.g. legal, government, regulatory risks, etc.)

Each risk in itself and/or combination of these risks can directly affect the deadline, cost and quality of the (PD³ ) process. Thus, the management of risks is prudent in order to satisfy projects' objectives in a complex CE multi-site project environment. The enabling agents of CE are essential ingredients to include in the whole risk management process ([37] Tseng et al. , 2003).

The development of the IRMAS(TM) caters for CE by taking into consideration the accumulative effects as a result of interdependency between tasks which cannot be tackled by traditional project management techniques. The shortcomings of traditional project management and the use of probability theories alone for predicting project durations have been reviewed in detail ([19] Kara et al. , 1999; [41] Zhou et al. , 2002). However, the research outlines that heuristics may be used to define the uncertainties associated with CE.

This research provides a structured and generic proactive approach to risk management of CE projects based on "Design WITH" concept, using the Australian/New Zealand Standard on Risk Management (AS/NZ 4360) as a framework ([2] AS/NZ Standard 4360, 1999). Additionally, in order to facilitate multiple-site locations, the system developed is compiled as a web-based portal in Java.

Commercially available software

Most of the existing commercially available tools for project risk management are mainly designed for risk analysis and assessment. Several commercial-off-the-shelf (COTS) software for risk management were evaluated, and only a few of them were found to have some features to be utilised. For instance, the available COTS software lacked capabilities to support project managers in a MSMP CE environment to identify, analyse and mitigate risks during the life cycle of the project. There are no COTS software available that can capture and reuse the lessons learnt from previous projects, case studies and best practices, to utilise and share the previous as well as existing knowledge and experience within the companies.

Also COTS software lacks in assistance to project managers in identifying the gaps, conflicts and propagation of risks during the (PD³ ) cycle of the projects. As mentioned in the previous sections, activities in CE projects' have high degree of interdependencies and complexities in nature which have not been covered by available COTS software. Some of the COTS software evaluated are Risk Track, TRIMS, Risk Advisor, Risk ID Pro, Crystal Ball Premium, Decision Programming Language, FMEA Pro, Data Suite, Project Risk and Contingency Analysis Risk, @Risk, Top Rank, Risk View, Scan$ Data Desk 6.1, Pert Master, Precisiontree, Risk Optimizer, Blaze Advisor and Bayes Engine ([41] Zhou et al. , 2002; [21] Kayis et al. , 2004; [40] Zhou et al. , 2003).

Development of IRMAS(TM)

The IRMAS(TM) software has been developed with a contingency for MSMP CE projects. The software systematically integrates an organisation's culture with project, product and process related risks by utilising knowledge engineering.

IRMAS(TM) is composed of four development stages; namely contextual establishment, risk identification, risk analysis and risk mitigation which resulted in development of several modules. The snapshot is shown in Figure 1 [Figure omitted. See Article Image.]. Knowledge elicitation is a critical part of IRMAS(TM). Its role and function can better be understood by the following summarised requirements to:

determine the dynamic nature of manufacturing environments and CE projects;

establish mechanisms for mapping and storing risks related to project, product process and organisation all through the life-cycle of the project;

establish mechanisms for mapping risks related to customers and suppliers thus covering the extended enterprise;

establish mechanisms by developing profiles for customers and suppliers in order to utilise previous and current knowledge;

establish mechanisms to continuously update the knowledge to be gathered;

establish methodologies for assessing and analysing risks;

determine risk mitigation options for identified risk items through lessons learned, case studies, best practices and expert knowledge;

develop and implement a risk mitigation model aiming at mitigating the source of risk and utilising the mitigation budget available for the project; and

develop action and mitigation plans for the users on the selected risk items.

This research paper mainly covers the first five requirements of IRMAS(TM) which correspond to "context establishment" and "risk identification" stages.

IRMAS(TM) system requirements

User specifications (US)

IRMAS(TM) is aimed at providing a framework for a decision-support tool to accomplish effective risk management in "Design WITH" project environment by considering the following principles:

- global perspective;

- forward-looking view;

- open communication;

- integrated management;

- continuous process;

- shared product vision;

- team work; and

- extended enterprise.

Accordingly, besides the "expert opinion" of the research group, several project managers of an aerospace company who is also the industrial partner in this research project derive the US for IRMAS(TM). US were derived from the list of requirements compiled from the potential users of IRMAS(TM). These requirements were prioritised in order of importance by the users. In total, 31 essential requirements were incorporated in the IRMAS(TM). Table I [Figure omitted. See Article Image.] shows the "most important" of the ten requirements indicated by the managers. As it is seen from Table I [Figure omitted. See Article Image.], the users were also asked to indicate:

- the life-cycle of the project (e.g. planning, design and manufacturing (operation), delivery and customer support).

- phase(s) of the risk management of the project that each requirement is essential.

Operating scenarios (OS) and originating requirements (OR)

OS were prepared in order to establish a framework on how the user will utilise IRMAS(TM). They are the list of potential user-system interactions throughout the project's life cycle. Some examples are given in Table II [Figure omitted. See Article Image.].

OR are the list of functionalities required for IRMAS(TM) to meet US to facilitate OS. Based on the OR, the level of difficulties in implementing the US can be determined. The level of difficulties to implement the US and the importance level of the requirement governed the originating order of the system requirements. In addition, the OR analysis addresses the implementation testing and evaluation of IRMAS(TM). In total, 131 OR were incorporated in IRMAS(TM). Table III [Figure omitted. See Article Image.] shows some examples of OR considered, with the parent links to US and OS.

OR were considered for all these stages of IRMAS(TM):

- system set-up;

- system start-up;

- knowledge warehouse;

- project planning phase;

- project operational phase;

- project closure phase; and

- user log-out.

The sections below discuss the modules developed for IRMAS(TM).

Context establishment

In each organisation, management has a major role in addressing and managing several risks inherited throughout the company. Hence, it is vital that contextual risks are identified in order to determine the systemic problems and company culture and applied across the entire board based on the relationships between various tasks. The "contextual" component of IRMAS(TM) defines organisational and user details, project objective, ownership, management support, regulatory requirements, nature of the project (e.g. build to print, derivative design, new design), type of project (e.g. commercial, military), schedule cut-off dates, estimated project budget, mitigation budget and government and/or regulatory authorities that needs to be complied. The dynamic and unique nature of manufacturing environment and projects are essential ingredients captured in this module.

The context establishment primarily sets the scene for the organisational nature of the company such as the project objective and management style or company culture. Every organisation has an inherent culture whereby the influences can be observed through most of the company. These factors determine the systematic problems which essentially do not change from one project to another. Therefore, context establishment module for IRMAS(TM) establishes the overall risk profile by assigning a weighting to the infrastructure of the organisation after the user's responses to a series of questions, covering the above-mentioned issues.

These questions are retrieved from the Expert Interview Facility (EIF); a database where all phase questions are stored and displayed to the users via the virtual workbench. The virtual workbench is to promote interactions with other project participants and facilitate communication. The workbench also allows the presentation of computed results in the form of a risk registry after each phase of the project is covered by the user. The context establishment module is built as a Java module that interacts with the risk identification and knowledge warehouse modules which will be explained in previous sections.

Risk identification

The "risk identification" component of IRMAS(TM) is designed to be sufficiently generic enabling applicability for numerous projects yet with the flexibility to capture critical details for future reference as a lesson learnt document. A more holistic approach to project management was taken with the incorporation of departmental head at the very initial stages of the project. Departmental heads such as quality assurance, design, tooling, manufacturing, business development, research and development, etc. are involved as early as the first phase (for example; requirements planning, conceptual design) thereby promoting higher levels of communication between various departments and hence reducing risks that are likely to occur. Consequently, risk that often come to fruition due to conflict of interests from various parties (for example; suppliers, partners, customers) are identified early with prevention in mind.

Risks considered in IRMAS(TM) were categorised into eight areas - schedule, technical, external, organisational, communication, location, resource and financial (Figure 2 [Figure omitted. See Article Image.]). The description of these risk types are as follows:

Schedule risk. In a plan of procedures for a specific project with reference to a sequence of operations that encapsulates the milestones, task dependencies, lead times, production planning, etc.

Technical risk. Is related to a professional trade involving mechanical, industrial or applied sciences. It includes design specific issues as well as manufacturing specific issues such as quality assurance, product/process design, technological know-how, innovation and technical support.

External risk. Is related to any issues with regards to any parties outside of the organisation (e.g. changes in customer requirements) despite the "Design WITH" customers as the core approach adopted. Furthermore, legal, government, regulatory requirements, etc. are considered as well.

Organisational risk. Is related to the management or administration personnel of the business. More specifically it is defined by the organisational structure, ownership, stakeholders, leadership and the organisation's culture.

Communication risk. Is the ability to effectively convey ideas and information within the company and externally to suppliers and customers. Communication encompasses language barriers, cultural differences and communication channels.

Location risk. Is the physical distance/barrier between two respective parties including their geographic location, proximity to each other, number of project sites and their size.

Resource risk. Is the available capabilities relating to supplies or support including material, labour, equipment and facility specific issues.

Financial risk. Is related to monetary receipts and expenditure. More specifically, it includes currency exchange rates, inflation, budget and costs.

The risk identification module in IRMAS(TM) is developed in Java and covers the complete product design development and delivery process within a MSMP CE environment.

The risk identification module utilises a database of questions covering all the above-mentioned issues which are potential risk items related to the nature of project, project environment, organisation, product, process partners, etc. The project life cycle is divided into six virtual phases where phases are concurrently run and some sub-activities and risk items of each phase overlap and get linked to the sub-activities of other phases all through the (PD³ ) process as follows:

requirements planning;

preliminary design;

detailed design;

manufacturing;

prototyping and testing; and

certification and customer support.

Each of the above phase is defined by several sub-activities and specific risk items classified under eight risk categories discussed above. The risk identification module of IRMAS(TM) is developed very elaboratively due to the expanse of the scope. It is linked to the knowledge warehouse module where customer and supplier profiles are constructed with the aim of increasing inheritance of intelligence. Product, project and process related issues are used in capturing information on the reliability, expertise, capability, capacity and compatibility of customers and suppliers.

Previous knowledge, engineering standards, best practices, case studies, industry procedures and lessons learnt on several completed CE projects are used to identify potential risk items through interviews with several functional managers and staff as well as information and data gathered from several documents and project contracts. More specifically, data collection at our industrial partner began with an exhaustive interview with the project (program) manager, manufacturing manager, Integrated Product Team (IPT) members, etc. An informal interviewing process with structured open-ended questions were used during the discussions. Several projects with successes and failures were chosen as subject matters in order to capture "what to do" "how to do" "why to do" as well as "not to do" scenarios. The information captured are summarised as follows:

- Motivations, risks and competitive pressures surrounding the organisation and project.

- Product complexity and technologies used.

- Project scheduling and concurrency.

- Team size and structuring, personnel, authorities and incentives.

- Relationships with partners, suppliers and customers, their organisational and cultural structure.

- Functional interactions and communication modes between teams in the organisation and with other teams in partner organisation.

- Methods and tools used to promote CE.

- Methods and tools used to mitigate identified risks.

- Barriers for success and key success attributes.

During the interviews, at least two researchers were present as to minimise if not eliminate the possible misunderstanding, misinterpretation, non-conclusion of points indicated by the participants, etc. The project managers of each case study or lessons learnt covered in IRMAS(TM) served as a contact person for the project, providing access to documents, company reports, organisation charts, planning documents and project schedules.

Several follow-up, face-to-face discussions with participants were carried out, supported by phone, e-mail, fax and regular mail correspondence. The knowledge capturing process is based on quantitative analysis in the background for risk identification facilitated by risks identified previously through expert opinion, interviews, case studies, lessons learnt, etc. (Figure 3 [Figure omitted. See Article Image.]) with the aim of maximising objectivity in risk analysis. More specifically, the magnitude of the risks identified is a product of its likelihood to occur and consequence where a five point exponentially increasing scale was used (Table III [Figure omitted. See Article Image.]).

The likelihood of a risk is the probability of an event destined to happen or the relative frequency of occurrence. The consequence of a risk is the outcome following the effect or result of an occurrence of the risk. For example, the likelihood of inaccurately estimating the milestones using simulation technique is rated as "medium to high" risk whilst the consequence may be rated as "low" because the inaccuracies of the estimate were known and hence taken into consideration in the schedule. The example in Table III [Figure omitted. See Article Image.] illustrates risk assessment in establishment of milestones based on how they are derived and how complete is the product design; both being in phase 1 - requirements planning.

The phase 1 requirements planning, outline the activities in the requirements (i.e. conceptual design) phase and are intended to be used as a preventative tool for risks that may occur in the preliminary design phase. It covers requirements related to milestone accuracy, customer, legal, financial, personnel, equipment, material, technological know-how, process specification, product specification, compatibility, location, communication, organisational structure complexity and ownership-related issues. Potential risk items may result due to not covering or not addressing them appropriately in the projects.

Figure 4 [Figure omitted. See Article Image.] shows briefly the type of risk, risk items and risk activities covered in three risk categories, namely schedule, external and technical risks. Accordingly, inaccurate schedule specification (schedule risk) may be due to milestone inaccuracy (sub activity) which could be a result of several "risk items"; for example, inaccurate schedule estimation, ill-assigned schedule responsibility, inaccurate reserve time estimation, existence of task dependencies (concurrencies), incomplete task definition and/or uninvolved department heads, etc. Similarly, incomplete technical product specifications (technical risk) may be a result of incomplete process specification, incomplete product specification and/or incomplete knowledge or know-how; all related to sub activities in the requirements phase. The risk items identified are incomplete conceptual design, incomplete product functionality evaluation, incomplete product performance evaluation, incomplete preliminary quality assurance and incomplete conceptual manufacturing process design.

In CE projects, in order to achieve "Design WITH" successfully, the knowledge management needs to be incorporated into the risk management process. Thus, the knowledge gathered about the customer (e.g. customer's tendency to make design changes, contractual requirements, legal requirements, culture, compatibility, etc.) as well as material and tooling (equipment) suppliers (e.g. supplier's price conformance, quality conformance, delivery conformance, technical support, logistics, culture, compatibility, etc.) are integrated in the risk identification process in IRMAS(TM).

Risk items and sub activities identified in each risk category interacts with other risk items and sub activities and increase/decrease the occurrence (likelihood) of resultant risks in each phase and their effect may propagate to other phases with varying impacts on the project. For example, estimating a defined sub activity such as milestone inaccurately not only may lead to an inaccurate schedule specification (schedule risk), but also may lead to "inaccuracies in personnel requirements" (resource risk) as well as "incomplete technical product specifications" (technical risk) and "ineffective communication" (communication risk). Table IV [Figure omitted. See Article Image.] briefly summarises the type of risks (round shape), risk items (rectangular shape) and sub activities (double-line rectangle) used in phase 1 of IRMAS(TM). Figure 4 [Figure omitted. See Article Image.] shows the links between three risk types; namely schedule, technical and external risk leading to risk events such as; inaccurate schedule specification, incomplete technical product specification and incomplete customer requirements analysis, respectively, (eclipse shapes). The risk types, risk items and sub activities are shown as round, rectangular and double-lined rectangular shapes, respectively, for phase 1 - the requirements analysis. The customer information may be retrieved from the knowledge warehouse customer profiles and shown as a dashed rectangle in Figure 4 [Figure omitted. See Article Image.]. Similarly, supplier profiles are retrieved from the knowledge warehouse in the later phases of the project.

Most of the sub activities and/or risk events of each phase are linked to other sub activities/risk events of other phases since phases are considered to run concurrently. Some of the links between phase 1 (requirements planning) and phase 2 (preliminary design) sub activities/risk events are summarised in Table V [Figure omitted. See Article Image.].

Each of the risk items is represented by a question in the database of EIF of IRMAS(TM) and displayed to the user via the virtual workbench. For example, in phase 1, a total of 52 questions were used with more than 200 links connecting each risk item or risk event to another one within phase 1 and other phases, if necessary. The user's responses to each risk item, in terms of likelihood of it to occur and its consequence if it occurs are stored in risk identification module to be analysed in the risk assessment module. The same approach is used to identify the risk items and sub activities leading to different type of risks for each phase. Risk inheritance from previous phases is also considered in all phases of IRMAS(TM) if the identified risk(s) are not mitigated before the next phase starts.

Knowledge warehouse

The knowledge warehouse is a collation information captured from generic engineering know-how, lessons learnt (in-depth internal expertise), case studies (internal and external case-based knowledge), best practices (external benchmarking) and engineering standards (e.g. Australian/New Zealand Risk Management Standard) (Figure 3 [Figure omitted. See Article Image.]). The access to such a knowledge means that the tool is capable of enabling the use of past successes and failures captured to minimise risks in project management. Thus, it is a useful tool for informing project managers on several aspects of (PD³ ).

Case studies

Case studies based on specific projects were primarily used either through interviewing or capturing information and identifying critical success and failure factors. A database of risk items identified was populated with a summary of both internally and externally used case studies. A description of the risks including risk event drivers, mitigation strategies implemented, risk consequence and likelihood constitute the database of case studies. There is additionally a provision for multiple case studies to be assigned to the same risk item. Therefore, the user of IRMAS(TM) will be able to locate the number of occurrence of the same risks in several projects both within their organisation and externally. Furthermore, case studies can be accessed via a collaborative environment or virtual library during the risk mitigation stage.

Lessons learnt

The knowledge elicitation of lessons learnt is the extension of case-based studies which capture in-house past experiences in more detail. The success of a project can be enhanced by considering successes and failures of previously completed projects. In other words, a success factor can be derived from historical lessons learnt, otherwise previous mistakes can be repeated leading to failures. Furthermore, the lessons learnt also help identify location of critical risk items or risk items which are identified based on success factors from lessons learnt and may be used to update module 1, risk identification module in the future. The gathering of information from several interviewees on the same project served to validate and provide increased credibility of the findings.

Additionally, it also captured information from different aspects of the project depending on their specific role in the team, background, experience and personality. For instance, interviewing members from design and manufacturing often highlight opposing opinions but both prove beneficial at different phases of the project. Additionally, archives of the risk assessment documentation analysed were used to establish the lessons learnt. IRMAS(TM) is designed to generate lessons learnt and build onto the knowledge warehouse on completion of each project. Similar to case studies database the lessons learnt database can be accessed either during the risk mitigation stage or through the virtual library.

Best practices

Benchmarking of other successful organisations is a tool often used in manufacturing to strive for excellence. Various best practice web sites are used to gather information with respect to successful risk management, readily available from an array of companies. Hence, the application of best practices in IRMAS(TM) not only provides an avenue for transferring excellence from several sources into the organisation, but also serves to populate the database with respect to identification of risk items and mitigation strategies.

Profiles

The process of risk identification encapsulates enormous areas of activities. However, one area where the repetition of knowledge capturing can be greatly reduced is background information on previous and current project partners. Therefore, one of the knowledge elicitation techniques used was to set up profiles of customers and suppliers which are stored in the knowledge warehouse. The profiles not only provide a reliable means of accessing previous history on customers and suppliers, but they also enhance the inherited intelligence of IRMAS(TM). The captured customers profile information includes, for example, preferred standards, geographical location, logistics, communication and cultural compatibility, reliability (on-time payments) and technical support required from them.

Supplier profile information captured includes geographic location, logistics, communication compatibility, technical support given, conformance to quality, delivery and price requirements. The system is constructed so that, profile information about the relevant partner is readily incorporated during the risk assessment process. The user is periodically prompted to confirm the relevance of the information in order to maintain an updated database of knowledge.

Engineering and quality standards

Australian/New Zealand Risk Management Standard and other engineering, quality standards, company procedures, etc. may be incorporated into the development of the various phases of the system. For example, the conceptual design phase incorporates the organisation's engineering standards for preliminary design review, critical design review as well as validation, facility audit and bill of materials. Accordingly, IRMAS(TM) is not just about the integration of "engineering" functions but covering the "product value stream" including suppliers, customers, purchasing, service and support departments, etc. in order to cover "Design WITH" and "ECN" paradigms mentioned in a previous section.

Risk assessment

A risk query mechanism was formulated through causal diagram representation of risk items and imposed on the concurrent phases-based process model to collate risk interactions and evaluate quantitative risk data and imposition of qualitative criteria on data through analysis techniques used. The risk evaluation consists of decision support systems using the analytical hierarchy process (AHP) ([35] Saaty, 2001) and Bayesian belief network (BBN) ([32] Press, 1989) to utilise previous knowledge to evaluate the real consequence and likelihood of risk items, respectively. Both techniques process the risks through the defined paths, capturing concurrency between tasks as well as further reducing user-related subjectivity using default values captured through expert knowledge, interviews held with IPT members and historical risk behaviour. The knowledge warehouse of IRMAS(TM) which is composed of case studies, best practices, engineering, quality standards and lessons learnt support defining the "default values" in order to minimise the user-related subjectivity. Whereas, the ultimate decision rests with the user whether to accept or not to accept the default values set by the system.

Risks worth investigation are highlighted through analysis due to their high chance of occurring or high potential impacts based on previous knowledge incorporated into the decision support modules. This analysis then provides a possibility for being pursued further in detailing risk items in detail, leading to the risk mitigation function.

All identified risks with their likelihood, consequence and magnitude (likelihood times consequence) values are processed in the risk assessment module of IRMAS(TM) through the risk paths given by the interactions between other risk items in each phase. The dynamic nature of manufacturing environment is taken into account and the likelihood of risks which are not mitigated are inherited by the following phases.

Risk assessment not only provides an automated and reliable mathematical cross-reference in calculating the risk magnitude, but it is sufficiently flexible to allow the user to also provide a manual assessment of the identified risks. The system thereby is capable of increasing its level of intelligence by building onto the knowledge base via this interactive network.

Risk mitigation

The risk mitigation module of IRMAS(TM) prioritises the assessed risk and weave an avenue for alternate mitigation strategies. A collaborative environment interacting with the knowledge warehouse can be accessed using a search and retrieval tool. Priority for the mitigation actions is controlled by the risk magnitude and the threshold value required for that specific item. Threshold values are determined by the users. A search facility is developed for access to various components of the knowledge warehouse including case studies, lessons learnt, best practices as well as generic engineering standards and procedures.

Furthermore, the selection criteria governing the mitigation plan is facilitated via associated costs and available resources and mitigation budget. An analytical model is developed to address mitigation plans ([1] Amornsawadwatana, 2003).

Validation

The full validation of IRMAS(TM) is successfully carried out in two large-scale new product development projects. It has already been decided to be deployed by a large international aerospace company and is successfully commercialized.

Comparative analysis between IRMAS(TM) and COTS tool for risk management

The risk management COTS tool chosen for comparative analysis with IRMAS(TM) is TRIMS. This is due to the similarity in the risk management process between the two systems. To facilitate the comparative analysis between TRIMS and IRMAS(TM) both systems are used to analyse an aerospace case study gathered from the industrial partner of this research. Prior to analysing the risks for the case study using TRIMS, the system needs to be configured. The configuration involves inserting the questions used in IRMAS(TM) and specifying the eight risks categories (schedule, technical, external, organisation, communication, location, resources and finance) considered in IRMAS(TM). Furthermore, it is assumed that categories and questions are of equal importance. Hence, equal weighting for the categories and questions are to be carried out. In order to ensure direct comparison between the two systems, the responses for the questions in IRMAS(TM) are translated into available responses in TRIMS. In this analysis, only responses yes, partial and no in TRIMS are used.

Additionally, the thresholds for the risk levels in TRIMS are set using the default values. Having configured TRIMS, the questions are given the equivalent responses to IRMAS(TM). Based on the responses to the questions in TRIMS, the overall degree of compliance for risk category is then automatically computed. Subsequently, the risk level for each category is determined according to the threshold value.

The degree of compliance computed by TRIMS needs to be translated into risk magnitude, in order to allow direct comparison with output from IRMAS(TM). The translation of the risk magnitude involves subtracting the degree of compliance by 100 per cent. In other words, risk magnitude is represented by the degree of non-compliance.

The results obtained from TRIMS indicate the magnitude of each risk category, whilst the results from IRMAS(TM) indicate the magnitude of each risk factor. To allow for direct comparison between the systems, the magnitude of each risk category needs to be calculated based on the outputs from IRMAS(TM). The computation involves firstly summing the magnitude of the risk factor in a risk category. The summed of the magnitudes is then divided by the maximum magnitude of the risk category. The process of calculating the magnitude of risk category can be expressed in the following mathematical equation: Equation 1 [Figure omitted. See Article Image.] where, RCM=magnitude of risk category, RFM=magnitude of risk factor, n =number of risk magnitude in the category.

Table VI [Figure omitted. See Article Image.] presents the results of TRIMS for direct comparison with results obtained from IRMAS(TM). The comparative analysis shows that the results obtained from both systems differ significantly, except for technical risk, where both systems indicate a risk magnitude of 23 per cent. In general, the discrepancies are due to the adoption of different risk analysis methodologies, as well as assumptions made during configurations. The results obtained with IRMAS(TM) provided more in-depth and accurate risk identification, assessment and mitigation options ([23] Kusumo et al. , 2004).

The risk analysis process in IRMAS(TM) involves both quantitative and qualitative judgements. The qualitative judgements consist of deciding on setting initial weighting in AHP pairwise comparison and the prior probabilities in BBN. However, unlike TRIMS, the qualitative judgements used in IRMAS(TM) are acquired from the available individual and organisational knowledge gathered, rather than solely relying on user's knowledge. Additionally, the acquired qualitative judgements are rigorously treated for inconsistency in AHP, thus minimising the errors in analysis of results. As for the prior probabilities, a proven methodology is adopted in deducing the qualitative judgements into posterior probabilities to indicate the risk likelihood. In addition, IRMAS(TM) provides more in depth risk analysis that is specific to the risk factors at different stages of (PD³ ) process.

Financial measurements

The magnitude of the risk likelihood and its consequence is validated against financial measurements, specifically costing data provided by industrial partner for both case studies. This is achieved by ensuring that the output from IRMAS(TM) is proportional to the costing data. Any risk items identified outside this criterion are re-assessed for their prior probabilities and the knowledge elicitation process re-visited using the Delphi technique from experts. The financial measurements show a good correlation between the IRMAS(TM) output and the costing data.

Results and conclusion

A comprehensive risk management tool, IRMAS(TM) is being developed with a contingency for MSMP CE projects with the aim of achieving "Design WITH" and "Engineering as a Collaborative Negotiation" paradigms. It is achieved by developing a knowledge warehouse module which enables the use of past successes and failures captured to minimise risks in project management. Thus, it is a proactive tool supporting project management activities. It is a web-based portal compiled in Java facilitating effective and a common communication platform between partners thus reducing or eliminating several risk items from occurrence all through (PD³ ) process of the project.

In this paper, mainly the contextual establishment and risk identification stages of IRMAS(TM) are covered. The identification and management of risks in CE projects is essential to ensure better resource utilisation and greater contribution of projects towards organisational goals. A holistic process of CE in MSMP environment is used to identify several potential risk items in each risk category, as well as their interaction within each and between risk categories. The core intelligence of the system lies in the knowledge warehouse enabling system to use the past successes and failures captured to minimise risks in managing projects. The inclusion of experts judgements in risk analysis is beneficial, since the existing systems and technologies are incapable of capturing several potential risk factors. In addition, the existing COTS tools do not have the required "human" intelligence to integrate these factors into quantitative indicators. The involvement of experts' judgements is also required to ensure that human is the ultimate decision maker, and to promote traceability in the analysis results.

Thus, the quality of the results achieved from IRMAS(TM) will depend on the commitment and awareness of future users to populate the warehouse. Although it is developed as being a generic software, apart from more than one-hundred best practices evaluated and included in IRMAS^TM , the lessons learnt and case studies in the warehouse module constitute of aerospace industry projects only. Inclusion of several lessons learnt and case studies from variety of industries will enhance its full potential.

The authors gratefully acknowledge the financial support of Co-operative Research Centre - Intelligent Manufacturing Systems and Technologies (CRC-IMST) Australia towards this project (SP4.2) and close collaboration of the industrial partner BOEING-HdH.

References

1. Amornsawadwatana, S. (2003), "Risk management in multi-site concurrent engineering projects", unpublished PhD dissertation, The School of Mechanical and Manufacturing Engineering, University of New South Wales, Sydney.

2. AS/NZ Standard 4360 (1999), Australian/New Zealand Risk Management Standard, AS/NZ Standard 4360.

3. Boehm, B.W. (1991), "Software risk management: principles and practices", IEEE Software, Vol. 8, pp. 32-41.

4. Carter, B., Hancock, T., Morin, J.-M. and Robins, M. (2001), Introducing Riskman Methodology - The European Project Risk Management Methodology, NCC Blackwell Ltd, Oxford.

5. CCTA - The Government Centre for Information Systems (1993), An Introduction to Managing Project Risk, HMSO, London.

6. CCTA - The Government Centre for Information Systems (1994), An Introduction to Managing Project Risk, HMSO, London.

7. Chapman, C.B. and Ward, S.C. (1997), Project Risk Management: Processes, Techniques and Insights, Wiley, Chichester.

8. Chapman, R.J. (1998), "The effectiveness of working group risk identification and assessment techniques", International Journal of Project Management, Vol. 16 No. 6, pp. 333-43.

9. Chapman, R.J. (2001), "The controlling influences on effective risk identification and assessment for construction design management", International Journal of Project Management, Vol. 19, pp. 147-60.

10. Conroy, G. and Soltan, H. (1998), "CONSERV, a project specific risk management concept", International Journal of Project Management, Vol. 16 No. 6, pp. 353-66.

11. Cooper, D.F. and Chapman, C.B. (1989), Risk Analysis for Large Projects: Models, Methods and Cases, Wiley, New York, NY.

12. Craig, R.D. (2003), "Calculated risk: a framework for evaluating product development", IEEE Engineering Management Review, Vol. 31 No. 1, pp. 38-44.

13. Durofee, A.J., Walker, J.A., Alberts, C.J., Higuera, R.P., Murphy, R.L. and Williams, R.J. (1996), Continuous Risk Management Guidebook, Carnegie Mellon University, Pittsburg, PA.

14. Fairley, R. (1994), "Risk management for software projects", IEEE Software, pp. 57-64.

15. Haque, B. (2003), "Problems in concurrent new product development: an in-depth comparative study of three companies", Integrated Manufacturing Systems, Vol. 14 No. 3, pp. 191-207.

16. Hertz, D.B. and Thomas, H. (1983), Risk Analysis and its Applications, Wiley, New York, NY.

17. Jaafary, A. (2001), "Management of risks, uncertainties and opportunities on projects: time for fundamental shift", International Journal of Project Management, Vol. 19, pp. 89-101.

18. Kayis, B., Ahmed, A., Reidsema, C. and Webster, O. (2003), "Knowledge management: an essential ingredient for learning organisations", Proceedings of Human Computer Interaction Conference, Crete, Greece.

19. Kara, S., Kayis, B. and Kaebernick, H. (1999), "Concurrent resource allocation: a heuristic for multi-project scheduling with resource constraints in concurrent engineering", International Journal of Concurrent Engineering: Research and Applications, Vol. 9 No. 1, pp. 64-73.

20. Kara, S., Kayis, B. and Kaebernick, H. (2001), "Modelling concurrent engineering projects under uncertainty", International Journal of Concurrent Engineering: Research & Applications, Vol. 8 No. 3, pp. 269-74.

21. Kayis, B., Zhou, M., Savci, S., Khoo, Y.B., Kusumo, R., Ahmed, A. and Rispler, A. (2004), "Development of an intelligent risk management system for minimizing problems in new product development", Proceedings of International Concurrent Engineering Conference, China.

22. Klein, J.H. and Cork, R.B. (1998), "An approach to technical risk assessment", International Journal of Project Management, Vol. 16 No. 6, pp. 345-51.

23. Kusumo, R., Ahmed, A., Savci, S., Kayis, B., Zhou, M. and Khoo, Y.B. (2004), "A comparative study of intelligent risk mapping and assessment system (IRMAS) with commercially off the shelf tools for risk management", paper presented at PCMM Conference.

24. Kusumo, R., Zhou, M., Khoo, Y.B., Kayis, B., Reidsema, C., Ahmed, A., Couchman, P., Webster, O., McPherson, D. and Rispler, A. (2003), "System framework for an intelligent risk mapping and assessment system (IRMAS)", paper presented at 9th International Conference on Manufacturing Excellence - ICME, Melbourne, 13-15 October.

25. Larson, N. and Kusiak, A. (1996), "Managing design processes: a risk assessment approach", IEEE Transactions on System, Man and Cybernetics, Part A, Vol. 26 No. 6, pp. 749-59.

26. Lu, S.C.Y. and Cai, Y. (1999), "Modeling collaborative design process with a socio-technical framework", Proceedings of the 6th ISPE International Conference, on CE, Bath, UK.

27. Lu, S.C.Y. and Cai, J. (2000), "STARS: a socio-technical framework for integrating design/knowledge over the internet", IEEE Computing, Vol. 4 No. 5, pp. 54-62.

28. Lutters, D., Mentink, R.J., Van Houten, F. and Kals, J.H.H. (2001), "Workflow management based on information management", Annals of CIRP, Vol. 50 No. 1, pp. 309-12.

30. Perry, J.G. and Hayes, R.W. (1986), "Risk management for project managers", Building Technology and Management, August/September, pp. 8-11.

31. Prasad, B. (1996), Concurrent Engineering Fundamentals: Integrated Product and Process Organisation, Prentice-Hall PTR, Upper Saddle River, NJ.

32. Press, S.J. (1989), Bayesian Statistics: Principles, Models and Applications, Wiley, New York, NY.

33. Project Management Institute (US) Standards Committee (1996), A Guide to the Project Management Body of Knowledge, Reproduced by Australian Institute of Management, Sydney.

34. Raz, T. and Michael, E. (2001), "Use and benefits of tools for project risk management", International Journal of Project Management, Vol. 19, pp. 9-17.

35. Saaty, T.L. (2001), Decision Making for Leaders - The Analytical Hierarchy Process for Decisions in a Complex World, RWS Publications, Pittsburgh, PA.

36. Smith, P.G. and Merritt, G.M. (2002), Proactive Risk Management, Productivity Press, New York, NY.

37. Tseng, M.M., Kyellberg, T. and Lu, S.C.Y. (2003), "Design in the new e-manufacturing era", Annals of the CIRP, Vol. 52 No. 2.

38. Wideman, R.M. (1986), "Risk management", Project Management Journal, Vol. 17 No. 4, pp. 20-6.

39. Williams, T.M. (1995), "A classified bibliography of recent research relating to project risk management", European Journal of Operational Research, Vol. 85, pp. 18-38.

40. Zhou, M., Kusumo, R., Ahmed, A. and Rispler, A.A. (2003), "A report on commercially off the shelf risk management softwares", Project SP4.2, CRC-IMST, Victoria.

41. Zhou, M., Nemes, L., Reidsema, C., Ahmed, A. and Kayis, B. (2002), "Tools and methods for risk management in multi-site engineering projects", Proceedings of International DIISM Conference, CD Rom, Japan.

   

Further Reading

1. Morgan, J.P. and Monczka, R. (1996), "Supplier integration: a new level of supply chain management", Purchasing, Vol. 120 No. 1, pp. 110-3.

Appendix

Corresponding author

B. Kayis can be contacted at: [email protected]

  

Illustration

Equation 1

  

Figure 1: Snapshot of IRMAS(TM)

  

Figure 2: The risk categories and their relationships considered in IRMAS(TM). Phase 1 - the requirements planning

  

Figure 3: Population examples of knowledge warehouse for Aerospace Manufacturing Company

  

Figure 4: Some of the risk items, sub activities, risk events and customer profile used in phase 1 with their interrelationships between schedule, technical and external risks

  

Table I: IRMAS(TM) user requirements ranked as "most important"

  

Table II: Examples of OS and OR

  

Table III: Example of quantitative risk identification

  

Table IV: Interrelationship of major risk items, sub activities and type of risks - phase 1 - the requirement planning

  

Table V:

  

Table VI: Translated risk magnitudes