The use of Monte Carlo simulation in quantitative risk assessment of IT projects

Abstract

Estimation of the likely time and cost to complete the project and in line with it, taking into account the likelihood of occurrence and severity of the risks' effect, is one of the main concerns that have busied the organizational project managers. On the other hand, the diversity and sensitivity of information technology risks have caused to proper risk management, bolder than other issues, influences these projects. Therefore, in order to describe the degree of potential consequences and probability of occurrence of incidents accurately, IT project managers benefit from quantitative assessment. One of the most effective tools for quantitative assessment and likely forecasting of risks is Monte Carlo simulation, which by generating random numbers, calculates the individual components of a project and determine the impact of each of them on project. In this study, we tried to offer the functional model of the impact of risks on performance indicators of information technology project and propose proper time and cost for completing the project under the study by doing a case study and use of software functionality of Primavera Risk Analysis in Monte Carlo simulation.

Headnote

ABSTRACT

Estimation of the likely time and cost to complete the project and in line with it, taking into account the likelihood of occurrence and severity of the risks' effect, is one of the main concerns that have busied the organizational project managers. On the other hand, the diversity and sensitivity of information technology risks have caused to proper risk management, bolder than other issues, influences these projects. Therefore, in order to describe the degree of potential consequences and probability of occurrence of incidents accurately, IT project managers benefit from quantitative assessment. One of the most effective tools for quantitative assessment and likely forecasting of risks is Monte Carlo simulation, which by generating random numbers, calculates the individual components of a project and determine the impact of each of them on project. In this study, we tried to offer the functional model of the impact of risks on performance indicators of information technology project and propose proper time and cost for completing the project under the study by doing a case study and use of software functionality of Primavera Risk Analysis in Monte Carlo simulation.

Keywords -risk management, quantitative risk assessment, Monte Carlo simulation, IT Projects.

Date of Submission: May 25, 2015 Date of Acceptance: June 25, 2015

I. INTRODUCTION

Uncertainty in estimating the time and the estimated costs of the projects is contemplated as a major challenge in the scientific managing the projects [1].Risks and uncertainty in general is led to various results which indicates the expected result [2] .Therefore, the risks of the project have to be recognized and quantified before any project starts, and finally in order to prevent from such happenings, appropriate strategies should be taken into consideration to minimize the risks [3].

In this regard, one of the most effective solutions to solve this dilemma is risk assessment. Risk management, the systematic application of management policies, procedures and processes related to the activity of analysis is the evaluation and the control of risks [1], which recognizes the risks and designs the strategies [4].

The evaluation of the risk is the main part of the risk management process which consists of the risk recognition stages, estimating the risk and evaluating the risk. In order to estimate the risks, quantified and qualified methods are used. In the estimation of the potential implications in order to describe the amount of quality and the possibility of the occurrence of events, from low, medium and high are used. In contrast, the quality estimations are used to show the consequences and probabilities using and data [5].One of the methods of artificial intelligence used in quantitative evaluation of the risk is the artificial neural network [6].

Monte Carlo simulation method used as a standard in PMBOK (2004) is introduced as one of the most used methods in the risk quality analyzing of the artificial intelligence. This method can model the phenomena into two parts: the determined factor (the activity of the project) and random factor (time and the activity costs). Some characteristic features, such as the probability distribution, average, certainty spaces and variance are being introduced in the random factor, and then the model is designed and they are taken into consideration for the forecasting. The Monte Carlo simulation provides a possibility to have a behavior like the random factor and as a result, it makes calculable the risk [7].

The present article is intended to take advantage of the field and library methods by focusing on the strengths of Monte Carlo simulation in assessment of the risk in this approach, and to make use of it in the quantitative evaluation of the risks of the project for Information Technology domain, and so to have a case study and use the Primavera Risk Analysis software in the quantitative analysis of such risks, and finally, evaluate the results.

II. RISK AND THE LACK OF CERTAINTY IN A PROJECT

On one hand, risk means the probability or the possibility of not meeting one's expectation, endangering or confronting danger, damage, decrease in the amount of earnings and loss [8] .On the other hand, risk is known as the recognition and the evaluation of the incidents and the probable factors which can cause positive consequences (chance) and negative consequences (dangers) [6].

Uncertainty refers to the events and the unrecognized factors and incalculable in a way that there is no possibility for the recognition and evaluation of it [6]. Therefore, the difference between these two concepts is that unlike uncertainty, the risk can be recognized and controlled. Fig. 1 shows the uncertainty and the risk of the different phases

At present, the available technology makes us familiar with some instruments estimating and controlling the probable dangers, and finally, it always keeps us far from it [9].

III. RISK MANAGEMENT

The purpose of risk management is the management of uncertainties and includes the identification of activities, risk assessment, monitoring and reducing the impact on the business. A proper risk management program with appropriate risk management and suitable strategies can reduce to the minimum the cost and stressful problems [10].

According to Bohm theory, risk management is a process that includes two main phases: the estimation of the risk (identification, analysis, prioritization) and risk management (planning of risk management, risk monitoring and corrective actions plan) are included in this management [10].

According to Fairly, the risk management includes the seven phases of the identification of the risk factors, estimation of the probability of risk taking, presenting some procedures for eliminating the identified risks and the revival of the organization after the crisis [10]. In sum, one can remember the risk management as a system that gives a directed order to the counter operations or the uncertainty of the probability design[9]. Fig. 2 shows the main cycle of the risk management [11, 12].

Risk management is considered as one of the most used issues today in managing the projects. Therefore, the institutes and the scholars throughout the world have carried out extensive researches and have presented some certain stages for analyzing the risks, and most of them follow the same process [13].

IV. METHODS OF ANALYZING THE RISK

In fact, through the risk assessment, the amount of efficiency and effectiveness of control methods can be identified, and some valuable data are furnished for decision making in reducing the risk, risks, improvement of control systems and planning for their reaction [10] .The quantitative assessment of the risk requires calculating two risk factors: the severity of the outcome of the occurrence and the probability of its occurrence. There are three methods to calculate the probability or the severity of the occurrence [10]:

- Quantitative methods resulting in a certain number

- Qualitative methods which is the result of a special quality in the risk field

- Semi-qualitative methods with the risk matrix to be used for most of them

In assessing the qualitative risk, the probability of a certain event and its consequences are calculated or estimated, and then the numeric criteria is used to judge the acceptability of the risks [10].

In the quantitative analysis of the risk, the whole project is simulated and the effects of each of the critical risks on the time and the overall costs will be studied [13] . In this analysis, the project activities in the scheduled plan and the risk programs are all available. The relation between the risks and the project activities are determined in this stage. This relation determines how the time and costs of each of the activities vary in accordance with each critical risk. Therefore, each of the activities can be probable, and a distribution function must be determined for it, and it is highly important to notice in selecting the distribution probability [13].

V. MONTE CARLO SIMULATION (MCS)

This method is a statistical technique asserting the critics that the risk in evaluating the lack of certainty of the project is of crucial importance [14] . and the origin of this analysis is Pert. The time effect and the costs of each risk in every activity is determined by collecting the data in this method [13], and the behavior of the risk factors in the time period of [t,t+Δt] by the supposed probable distribution function determined, random numbers are obtained [15], and finally through simulation one can determine the time and cost deviations [13].

If several non-conclusive variable that any large number of possible values can be provided as input variables in the economic evaluation of a project is considered, in practice the possibility of applying the analytical methods for the risk assessment does not exist.

In such a case by picking up the n samples from the desired community, an estimation will be deducted from the statistical parameters of the sample population, and the total population parameters will be simulated.

The Monte Carlo simulation technique is based on this. This technique simulates the available uncertainty in the model output. This uncertainty is caused by changing the input variables coming into existence because of different factors [16].

Fig. 3 shows the process of the simulation test of Monte Carlo. With random numbers generated, the simulation is carried out and the output is achieved [6]. By repeating the cycles of simulation with a large number, the results are closer to reality [7]. With the analysis and assessment of risk, the likelihood of occurrence of output is estimated. Selecting the distribution function fitted to the data input is the most critical stage of this process [6].

If the risk management cycle is divided into two General phases (identification and analysis) and (the evaluation and control) of the risk, Monte Carlo simulation fits in the second phase of the cycle.

In this way, firstly by identifying, analyzing and categorizing the risks of the project, the process of the risk management is initiated and in the second phase, the team of the project calculates the minimum, maximum and the average of the risk probability in n repetition by considering the fitted distribution of the project [11]. The weak point of this technique is that the relationship between the uncertainties of the project is not considered, and the project is analyzed and assessed according to each risk independently [14].

Monte Carlo calculations are easy and the accuracy with each round gets more by repeating, but the speed of the recovery is very low so that with improving the degree of accuracy the round numbers of the simulation are increased. Therefore, this process decreases the response speed of this method [17].

VI. RISK MANAGEMENT IN IT PROJECTS

even though IT is more reliable, faster, and cheaper, the costs, the complexities and the risks of information technology projects also continue to increase. Many organizations outsource the project to reduce the risks in IT projects, and only 30 percent of the companies are busy at managing such project services [18] .Most of these ventures can be related to the weakness of scientific human resources, the exclusiveness of some of the required resources, security and data maintenance and lack of adequate regulatory mechanisms in the implementation of information technology projects, etc [19, 18]. The magazine of the Information Week (1996) and the Info security News (1997) have achieved interesting statistics by asking "which problems are very risky in IT?" Combining the results of these two magazines, the lack of electricity, communication outage, disruption in the comprehensiveness of data, computer viruses, random errors, misuse of licenses by clerks, natural disasters, unauthorized access to computing resources, external intrusion, destruction of data and unauthorized access to the user names of others are among the most important risk areas for information technology [20].

VII. USING MONTE CARLO METHOD IN QUANTITATIVE EVALUATION OF THE RISKS OF PROJECTS FOR RUNNING THE ELECTRONIC TRADING IN KALAPORT TRADING COMPANY

In early 2014, Kalaport international trading company managed to implement optimal management system and portal content in order to run the electronic trading system aiming at sharing its own organizational information. Therefore, a team including critics, experts and managers of IT were given the mission.

This project like the other IT projects consisted of 5 phases of the process project management [21]: The Initiation Phase (X1), The Scheduling Phase (X2), The Performance Phase (X3), Monitoring and Control Phase (X4), and the End Phase (X5). At present, the initiation phase (X1) and the scheduling phase (X2) have come to end, and the managing team of the project intend to estimate the time and cost of the project in order to finish the performance phase. First, it is estimated that the required time considered for the completion of the phase (X3) is one year and an expense beyond 600,000,000 Rials.

The first step in estimating the precise time and costs of the phase X3 is the identification of the risks affecting the project by means of which the direct effects of each of them on the functional features of the project can be identified. According Table 1. ,the team of the project identified a list of the risks, the probability of the occurrence, the severity of the effects of each of the time periods, the efficiency of the project and the amount of the damages.

The next step is the use of Monte Carlo method in risk analysis and quantitative evaluation, and the study of their effectiveness on time and cost of completing the project. This phase of the project is carried out by Primavera Risk Analysis from Oracle Enterprise, which is one of the most widely used software in the area of Monte Carlo simulation and risk analysis. As it is indicated in figure 4., the listed risks in Table (1), along with additional information, in the part related to the recording of the software risk of Primavera Analysis Risk has been entered.

In the recording of the risk, the damages of phase X3 are placed on the triangle to evaluate and make use of the costs through a three-point estimation: optimistic damages, probable damages and cynical damages. To run the Monte Carlo simulation, the number of repetitions to simulate is considered 500 rounds. In order to receive the graphic outputs, the Show Distribution Graph option is activated. After the 500-round repetition of simulation is over, all the output information are indicated in a window titled Distribution Graph. The result of the Monte Carlo simulation or the probability of completion of phase X3 is indicated in fig. 5.

AS it can be viewed in the above fig., the horizontal axis shows the whole time dedicated to the completion of the phase X3, and the vertical axis shows the percentage probability of the time required for the completion of this phase. This diagram shows that the precision of the preliminary estimation is much lower than the required time for the completion of the phase X3 in the appointed time, and only with a probability of 37%, the phase X3 can be completed at the appointed time. Also with a probability of 80%, this phase of the project can be completed within 580 days. The output of this software indicates a three-point time estimation of the project. That is a cynical, probable and optimistic estimation.

Viewing optimistically, the minimum number of the days for the completion of this phase is 413 days, 531 days is required if the estimation is viewed with probability and through a cynical estimation, 654 days are needed. As can be seen in fig. 6, the cost tab in the output diagram shows the distribution of the cost probability. The diagram resulted from this software is indicative of this matter that the preliminary estimation of the required costs for the completion of the phase X3 can only cover 44% of this phase. In addition, with an 80% confidence and with an expense of beyond 6,639,339,345 rials, one can overcome the damages resulted from the probable risks, and to finish this phase of the project successfully.

The output resulted from this software shows a three-point estimation of the costs which are cynical, probable and optimistic in the successful performance of the project. Viewing optimistically, the minimum amount of the costs estimated in carrying out this phase is 4,625,114,233 rials, and 6,082,568,249 rials in time of probability and 7,527,797,854 rials when it is cynically viewed.

CONCLUSION

with the advent of new information technology and the need to respond and the synchronization of these organizations with this technology, the decision making within the scope of the project and information technology-based systems, it is inevitable not think of the risks and the existing dangers. Hence the necessity of implementation of risk management becomes more important than before. Using the appropriate manner of the risk management can help the IT project managers in identifying the factors involved on the deviation from the forecast cost of the project at the time of delivery and delay. In this project, the Monte Carlo simulation method was used to assess the risks involved in the portal enterprise of the trading company and to make use of it as a means of accuracy in estimating the time and budget required for the full implementation of the project. With respect to the results acquired from the quantitative assessment of the risks, one can assert that the initial estimates of time and cost of the project is just fitted to do half of the project, so it is necessary to reconsider and investigate the effects of probable risks.

References

REFERENCES

[1] D. W. North, "Limitations, Definitions, Principles and Methods of Risk Analysis," OIE Review of Science and Technology. Epiz, vol. 14, no. 4, pp. 913-923, 1995.

[2] G. Mahdavi and K. Godarzi, "Providing an artificial neural network to predict systemic risk with the macro-economic variables (Case Study: Saipa)," Economic Bulletin, pp. 219-237, 1389.

[3] D. Bolles, A Guide to the Project Management Body of Knowledge(Fifth Edition, Project Management Institute: Project Management Institute, 2013).

[4] Y. Tari Verdi and Z. Damchi Jelodar, "The relationship between risk and performance of the company," Journal of Financial Accounting and Auditing, vol. 4, no. 14, pp. 43-62, 1391.

[5] S. Avalin Chahar Soghi, M. Dostari, A. Yazdiyan and S. A. Mahdavi Ardestani, "The use of artificial neural networks in information security risk assessment," Journal -research electronic and cyber defense, vol. 1, no. 4, pp. 23-33, 1392.

[6] P. D.-I. J. Schwarz, "Implementation of artificial intelligence into risk management decision-making processes in construction projects," 3 3 2015. [Online]. Available: http://www.unibw.de/san/bauv8/veroeffentlichungen /.

[7] M. Shahbaznia and S. A. Taleghani, "Monte Carlo simulation analysis of the project costs," in Sixth National Congress of Civil Engineering, Semnan, 1390.

[8] G. Shahrokhi, "An overview of banking risks, risk management and risk-based supervision," Bank and finance, vol. 78, pp. 20-24, 1385.

[9] H. Zand Hesami and A. Savoji, "Risk Management in Supply Chain Management," Journal of Management development, vol. 9, pp. 37-44, 1391.

[10] l. Niyakan, "What is risk management?," news-Education magazine World News of Insurance , vol. 186, pp. 16-26, 1392.

[11] T. Nemuth, "Practical Use of Monte Carlo Simulation for Risk Management within the International Construction Industry," in 6th International Probabilistic Workshop , Darmstadt, 2008.

[12] H. B. Peter, "RISK MANAGEMENT: PROCEDURES, METHODS AND EXPERIENCES," RT&A # 2(17), pp. 79-96, 2010.

[13] A. Alam Tabriz and E. Hamzei, "Assessment and risk analysis of the project using the integrated approach to risk management standards PMBOK and techniques RFMEA," Journal of Industrial Management Studies, vol. 9, pp. 1-19, 1390.

[14] K. Rezaie, M. Amalnik, A. Gereie, B. Ostadi and M. Shakhseniaee, "Using Extended Monte Carlo Simulation Method for the Improvement of Risk Management: Consideration of Relationships between Uncertainties," Applied Mathematics and Computation, vol. 190, pp. 1492-1501, 2007.

[15] K. Peykarjo, N. Norollahi ? S. Behnam, "Measure the risk of corporate assets by using value at risk," Economic Bulletin, pp. 195-221, 1388.

[16] H. Badiei and M. Yousefi, "Estimation of optimal management of financial risk and economic systems, industrial and mining using Monte Carlo simulation," Financial engineering and portfolios management, vol. 2, pp. 99-117, 1389.

[17] A. Salami, "Review of Monte Carlo simulation method,"Economic Bulletin, vol. 3, no. 1, pp. 117-138, 1382.

[18] G. Jamali and M. Hashemi, "Assessment of the relationship between the risk factors in the IT projects of Bushehr Melat bank by using fuzzy Dymtl techniques," Information Technology Managment, vol. 3, no. 9, pp. 21-40, 1390.

[19] M. Adineh and R. Shojayi, "Risk management in IT projects: A case study of electronic information management thesis," 2009. [Online]. Available: it.behdasht.gov.ir.

[20] J. W. Meritt, "A Method for Quantitative Risk Analysis," in 22th National information system security conference, Arlington, Virginia, USA, 1999.

[21] M. Shami Zanjani and N. Farzane Kondori, "Provide a conceptual framework for the use of the mechanism storytelling project lifecycle management," Information Technology Managment, vol. 5, no. 3, pp. 83-104, 1392.