A KNOWLEDGE SYSTEM PROPOSAL WITH WEIGHTED GOAL PROGRAMMING APPROACH FOR MULTI-CRITERIA DEA MODEL AND AN APPLICATION

Abstract

In today's competitive environment, enterprises should use their resources correctly; they should continuously improve themselves and work efficiently. It is important to evaluate the performances of the units under the same conditions in enterprises according to each other, to see the current situations and to determine appropriate improvements in necessary points. One of the commonly used approaches to performance evaluation is Data Envelopment Analysis. Many approaches have been developed for the Data Envelopment Analysis model, and Goal programming using in multi-objective decision making solutions approaches is one of them. Goal Programming gives decision-makers the opportunity to evaluate many objectives together in the decision-making process. In this study, classical Data Envelopment Analysis and weighted goal programming approach for multi-criteria data envelopment analysis model was applied in the evaluation process of the projects worked in an automotive supplier industry. A knowledge system has also been proposed in order to evaluate the effectiveness of the projects periodically and to include new projects or conditions into the evaluation.

Headnote

Submission: 2/8/2020

Revision: 5/13/2020

Accept: 6/3/2020

ABSTRACT

In today's competitive environment, enterprises should use their resources correctly; they should continuously improve themselves and work efficiently. It is important to evaluate the performances of the units under the same conditions in enterprises according to each other, to see the current situations and to determine appropriate improvements in necessary points. One of the commonly used approaches to performance evaluation is Data Envelopment Analysis. Many approaches have been developed for the Data Envelopment Analysis model, and Goal programming using in multi-objective decision making solutions approaches is one of them. Goal Programming gives decision-makers the opportunity to evaluate many objectives together in the decision-making process. In this study, classical Data Envelopment Analysis and weighted goal programming approach for multi-criteria data envelopment analysis model was applied in the evaluation process of the projects worked in an automotive supplier industry. A knowledge system has also been proposed in order to evaluate the effectiveness of the projects periodically and to include new projects or conditions into the evaluation.

Keywords: efficiency, data envelopment analysis, goal programming, knowl edge- system

1. INTRODUCTION

On account of the fiercely competitive atmosphere, the businesses today need to operate productively and follow closely the operations to oversee whether their limited resources are effectively used or not. The best course of action for the businesses to assess their existing situation is to compare the efficiency across the specific units or processes producing similar outputs using similar inputs. One of the approaches to be applied for this purpose is Data Envelopment Analysis (DEA).

DEA is a non-parametric methodology which is applied to analyze the relative efficiencies of production units manufacturing similar outputs using similar inputs in cases when there is multiplicity of inputs and outputs. DEA is developed without regard to the preference structures of the decision-making units.

Another, just as important, aspect is that the efficiency rate can be close to or on the frontier in the conventional DEA when one or more input and output values are multiple. This situation makes the interpretation difficult. For this reason and also in order to integrate the preferences of the decision-making units into the analysis, the solutions can be offered by applying the Goal Programming (GP) approach to the process of benchmarking the efficiency.

As an extension of linear programming, the GP ensured the attainment of the best satisfying solution from among the satisfying solutions rather than optimizing the goal in the solution of decision-making problems with multiple goals. The studies including the GP and DEA were successively addressed in this part in conjunction with the findings obtained from the review of the relevant literature.

By using the lexicographic GP approach, the study published by Bal and örkçü (2007) worked out a multi-criteria DEA model which was used to measure the homogeneity of the weight distribution. Also, through a case study, the new model, while promoting the homogeneity of input-output weights, was proved to have the same relative efficiency as the conventional DEA. The study proposed the GP for multi-criteria DEA (Goal Programming Multi-criteria Data Envelopment Analysis-GPMCDEA).

The study conducted in 2010 by Bal and örkçü (2010) dealt with the appraisal of cross-efficiency, and proposed the GP models to be applied in the second stage of the cross-efficiency appraisal. With certain adjustments, the study proposed the GPMCDEA and GPDEA-CCR for use in the cross-efficiency appraisal, and through three quantitative examples, the understanding of the proposed model was reinforced.

The study carried out in 2014 by Izadikhah et al. (2014)showed how the DEA problems would be solved by converting them into multi-objective linear programming formulations. A case study was conducted to indicate how to perform an efficiency analysis which was based on data envelopment by applying the GP approach.

The study undertaken in 2016 by Zografidou et al. (2016) presented the best satisfying design for the renewable energy generation network of Greece considering the social, environmental and economic criteria and the EU targets and applying the weighted GP [0, 1]. Furthermore, the DEA was utilized in order to find the best satisfying network among the prospective network structures.

The model proposing the GP approach for the multi-criteria DEA was introduced by Rubem et al. (2017). The purpose of Rubem et al. was to expand the studies undertaken by Ghasemi et al. by identifying the deficiencies in the model suggested by Bal and örkçü and also to present a new WGPMCDEA model. A new theoretically congruent model which was known as the WGPMCDEA-CCR and which solved the multi-criteria DEA problem by using the weighted GP was developed to replace the GPDEA-CCR model as a response to the incongruities identified in GPDEA-CCR by Rubem et al. The study can be reviewed for more details. In this study, the model was quantitatively developed with real data retrieved from automotive supplier industry, and its results were interpreted. The weighted GP for the MCDEA (WGPMCDEA-CCR) was formulated from Equation (1) to Equation (9) below (Rubem et al., 2017).

Objective Function:

Constraints:

Here, h0; the relative efficiency of the appraised decision-making unit, d0; the rate of inefficiency of the appraised decision-making unit, dk; deviation value for the kth decisionmaking unit, yJk; the value of jth output for the kth decision-making unit, xik; the value of ith input for the kth decision-making unit, u,-; the weight of the jth output, vt; the weight of the ith input, di, dj"; the negative and positive deviations of the 1st goal, d, d2; the negative and positive deviations of the 2nd goal, d, d3; the negative and positive deviations of the 3rd goal, M; the variable corresponding to the linearization of the objective function for the minimization of the maximum deviation, g1,g2>9-z', the desired value of the 1st, 2nd and 3rd goals, A1,A2,A3 ; the weight of the 1st, 2nd and 3rd objectives in the objective function.

The study conducted in 2019 by Gholam et al. (2019)proposed a novel methodology for target setting in mergers by using the GP and inverse DEA. The proposed model was discussed in more detail through an illustrative example covering forty two banks in the banking sector.

Torres-Ruiz and Ravindran (2019) analyzed the supplier selection process in 2019 by applying the DEA and fuzzy GP models. The suppliers were assessed with the Malmquist productivity index approach. Applied to a business manufacturing automotive spare parts, the study stated that the proposed model was applicable to the appraisal, selection and follow-up of the supplier.

The purpose of this study is to develop a solution to the multi-criteria DEA model through the GP approach and to ensure an interactive assessment process within the business with this approach. In the study, the efficiency was first assessed with the DEA-CCR for six projects in an automotive supplier industry, and then, the study, focusing on the weighted GP multi-criteria DEA model, proposed a knowledge system to appraise the efficiency of projects and to be used instantly or periodically in the business.

In designing the knowledge system, the Weighted Goal Programming Multi-Criteria Data Envelopment Analysis (WGPMCDEA) model which was proposed by Rubem et al. (2017) was apllied. Goal values were designated for inefficient projects by interpreting the results obtained after the solution. Different alternatives were tested for the targets attached to each goal, and their results were interpreted.

2. DATA ENVELOPMENT ANALYSIS

DEA which is used to benchmark the relative efficiency across the businesses which share the same targets appraises the efficiency of the decision-making units through computation of the ratio of the weighted sum of outputs to the weighted sum of inputs by using the observed inputs and outputs. This approach is a comparison instrument. It facilitates the determination of the comparison partners by ensuring the selection of the most efficient business among its peers in the same sector (Ross & Droge, 2002).

Information on what the magnitude of the necessary change for each input and each output should be in the allotment and usage of resources is offered by the DEA to the business manager in order to upgrade the performance of the decision-making unit to the level of efficient decision-making units which are selected as the reference points (üstünda&gbreave;, 2009).

DEA was first applied by Farrell in his studies through the comparison of efficiency in the case of multiple inputs and a single output in 1957. In 1978 Charnes, Cooper and Rhodes developed a mathematical programming technique with the assumption of constant returns to scale in order to analyze the total factor productivity measured as the ratio of the weighted sum of inputs to the weighted sum of outputs, and established the CCR (Charnes, Cooper & Rhodes, 1978) model (Charnes, Cooper & Rhodes, 1978; Banker, Charnes, & Cooper, 1984) established the BCC (Banker, Charnes And Cooper, 1984) model under the assumption of the variable returns to scale in conjunction with the CCR model. Banker and Morey developed the 'Categorical DEA' in 1986 in order to analyze not only the comparison of the performance of the DMUs across their own peer group, but also across other peer groups. Subsequently, focusing on the practical use of certain key topics, talked about the DEA technique (Boussofiane; Dyson; Rhodes, 1991).

The below are the steps to be taken in the application of the DEA:

a) The selection of the decision-making units: At this stage, decision-making units need to be homogenous units performing the same task. They should take part in the same processes and be working under similar environmental conditions. If the number of inputs is 'm' and the number of outputs is 'p', the minimum number of decision-making units must be minimum 'm+p+1'.

b) Determination of inputs and outputs: For all decision-making units, inputs andj outputs must be selected from common factors and in a way to affect the relative efficiency. For all decision-making units, the number of inputs and outputs must be positive, they should be obtained correctly, be reliable and there should be no deficiency in the data.

c) Selection of the model: An output-oriented model can be developed in cases if the control over the input is limited whereas an input-oriented model can be designed in cases when the control over the output is limited. If there is orientation towards neither outputs nor inputs, then the cumulative models must be selected.

d) Interpretation of the results: An efficiency ratio between 0 and 1 is calculated for each decision-making unit. The decision-making units whose efficiency rate is 1 are deemed as efficient whereas the decision-making units whose efficiency rate is below 1 are not relatively efficient. The efficiency scores talk about the distance from the efficiency frontier.

In general, the CCR Method, BCC Method, Cumulative Method and GP Approach are used in the DEA.

3. GOAL PROGRAMMING

Goal programming is an analytical approach dealing with the decision-making problems in which all characteristics of the goals are well-defined and decision-making unit is engaged with the minimization of the unattainable part of the goal (Türko&gbreave;lu, 2017). The GP which is a deterministic and multivariate model ensures the selection of best solution among the alternates rather than the goal optimization.

When solely an objective function is maximized or minimized in linear programming, the minimization of the deviations of multiple objectives from goals is the primary purpose in the GP. In the GP model, goal can be defined as an objective to attain a desired level whereas the objective is expressed as the reflection of a general desire by the decision-making units (Ignizio, 1985). For the solution to make sense, the goals, targets and constraints of the decision-making unit must be specified correctly. An achievement function is formulated for each goal, and a solution to minimize the deviations from these achievement functions is sought (Hillier & Lierberman, 1995).

The GP was first introduced by CHARNES, COOPER and FERGUSON in a study on the analysis of the salaries of business managers. Next, they developed a solution algorithm by clearly defining the GP in 1961. This algorithm was further developed in 1965 by IJIRI and then by Lee and Ignizio (1976, 1985, 1972). In its chronological development, successively,

Constraints:

Here, fi(x); linear function of x (x 6 X), b; the desired goal value, x; the variables in goals, d( ; positive and negative deviations from the goals. Objective function is given in Equation (3.1) in the model. Objective function is the sum of negative and positive deviations from the goals and it needs to be minimized. As seen in Equation (10), if the goals are not attained on ft(x), then the deviations will have a value. Equation (11) indicates that the multiplication of positive and negative deviations will be 0. The value of one of the two deviations must be 0, the values for both deviations will be above 0, but both cannot differ from 0 simultaneously. The oldest and most common GP types used in the literature are below (Ignizio & Romero, 2003):

* The weighted GP also known as Archimedean GP

* The lexicographic GP also known as pre-emptive GP

* The minmax GP also known as Chebyshev GP

4. A CASE STUDY

This study was conducted in the plants of Nursan Cable Equipment Industry and Trade Corporation supplying the cables and cable equipment manufactured for wheeled land motor vehicles such as passenger cars, light and heavy commercial vehicles to be produced by the automotive industry (Polat, 2019). Main customer groups of the above company are categorized as projects whereas the product to be manufactured for supplying each project is the cable equipment. Even if their size and materials may differ, the cable equipment is assumed to be homogenous by virtue of being manufactured with the same mode of production. For the efficiency appraisal through the DEA, two inputs and three outputs are specified. Table 1 shows the relevant data:

a) y1: Percentage value of the monthly sales turnover. The actual numbers were not provided on grounds of confidentiality, and so percentage values were given.

b) y2: Working pace of operators as percentages on a monthly basis. The sophistication level of the product, the failure to supply materials on time, the interruptions due to the placement of urgent orders and private needs of the operators are key factors associated with the working pace of the operators.

c) y3: This output value defined as the customer satisfaction was calculated taking into account the customer complaints and shipment performance. If there is no customer complaint and shipment performance score is high, customer is deemed to be satisfied.

d) x1: The number of employees in the proj ect-based production was specified as the input value. For instance, sixty two employees were involved in Project A whereas 7 employees worked for Project F.

e) x2: The land area allotted for each project. The land area remains fixed as long as no new project is developed, no new tools, equipment and materials are provided or no methodical changes are made.

4.1. Measuring the Efficiency through the Conventional DEA

The purpose of the DEA is to appraise the projects at hand, to categorize the projects in terms of the level of their efficiency and to offer proposals in order to make inefficient projects efficient. Under the assumption of constant returns to scale, input-oriented mathematical model is formulated from Equation (14) to Equation (17):

Objective Function:

Here, ??; efficiency rate of the kth decision-making unit, ??; the weight of jth output, ??; the weight of ith input, ???; the value of jth output for the kth decision-making unit, ???; the value of ith input for the kth decision-making unit. The problem was solved via DEAP Version 2.1 and Lingo, and Table 2 indicates the results.

The results show that Project B and Project F are efficient, Project A is efficient at 0.745 level, Project C is efficient at 0.58 level, Project D is efficient at 0.50 level, Project E is efficient at 0.54 level. For inefficient projects, the goals were specified on how to make the inefficient projects efficient. Dark colored sections are relevant to the goal values whose existing levels need to be changed for the inefficient projects. For instance, in Project D, the working pace of operators should rise from 40% to 50%, customer satisfaction should grow from 74% to 86%, the number of employees should drop from 14 to 7, and the land area should fall from 414 m2 to 110 m2. If all these measures are taken, Project D will also be among the efficient projects.

4.2. The GP Approach for the DEA and A Knowledge System Proposal

In view of the dynamic atmosphere in the company, it is natural to observe the cancellation of certain projects or the start-up of new projects or the emergence of changes in the values of inputs and outputs over time. It is necessary to have a dynamic system which will analyze the efficiency taking into account all these issues. In order for the company to use periodically as a useful tool to enhance the management process, the efficiency rates of the projects of the company were computed with the GP approach by using the WGPMCDEA-CCR model proposed in 2017 by Rubem et al., and a knowledge system was designed. In the system devised by using the Solver Add-in in the MS Excel, it is possible to make calculations by identifying 12 projects and inserting data for maximum 3 inputs and 3 outputs. This case study was carried out by using 2 inputs, 3 outputs and 6 decision-making units.

Figure 1 indicates the general view of the knowledge system. It is possible to insert data manually to the cells which are accompanied by sections for adding explanations, and the system itself makes the calculations with no other interference or effort. In this respect, the objective function is the minimization of the sum of positive deviations of the goals.

The right-hand side values represent the goal values (?1, ?2, ?3). The value of ?1 can be maximum 1 inasmuch as the inefficiency rate can be maximum 1, and it is specified as 1 in this study. The value of ?2 appertaining to the objective of minimizing the maximum deviation is specified as 0.2 by the decision-making unit. The value of ?3 appertains to the objective of maximizing the sum of all deviations and is defined as 0.6 in compliance with the demand of the decision-making unit. The weight of the deviations (?) is specified as 'one' by the decisionmaking unit.

Constraints were added and objective functions were defined in the Excel Solver and inputted to the buttons with codes. Two solutions were stipulated in the system. The former gives the detailed solution of project-based efficiency results and weights whereas the latter computes simultaneously the efficiency of all decision-making units.

For having the detailed solution for a single decision-making unit, it is possible to see project-based results by inserting the number of the decision-making unit, that is, the project number, into the section 'n'. To illustrate, for the Project C which is numbered as 3, the detailed results are displayed in Figure 2. The efficiency rate is approximately 0.58. The value of the objective function is 2.66. The weight of monthly sales turnover (?1) is 0.082, the weight of the number of employees (?1) is 0,059. All constraints were satisfied in the solution.

The button 'Compute the Efficiency of All Decision-Making Units' is clicked in order to compute solely the efficiency ratio for the entire set of projects. When a quick solution is sought and a snapshot on the current situation is needed to be viewed, just executing the button 'Compute the Efficiency of All Decision-Making Units' will be sufficient. Figure 3 displays the efficiency calculation interface for all decision-making units and the results.

Certain divergences were observed between the result obtained using conventional DEA (CCR-I) model and the result found using the WGPMCDEA-CCR model. The solutions were compared in Table 3. While Project B is efficient in the conventional DEA model, it is efficient at 0.93 level in the WGPMCDEA-CCR model. Only Project F is efficient in light of the results obtained through the knowledge system. The reason for the similarity in both results is that the goal values of the WGPMCDEA-CCR were assigned by the decision-making units with the help of the results already obtained using the conventional DEA model.

Table 4 indicated the supposed goal values and improvement percentages which were computed via Excel Solver for each project on an individual basis. The Table presented the supposed and actual values of inputs and outputs of all projects and improvement rates in percentages (variations). The colored parts were described as the parts having variations in-between and needing improvement. For example, the efficiency rate of Project C is 0.58, the references selected for Project C are the Project B and Project F which are numbered as 2 and 6 and whose reference weights are 0.0121 and 1.296 respectively. To make Project C efficient, the number of employees should be dropped by 47%, the working pace of operators should be enhanced by 51% and customer satisfaction should grow by 45%.

Knowledge system proposed for the business using the WGPMCDEA-CCR model has a dynamic character and quite open to changes. The knowledge system will be quite instrumental in ensuring the allocation of the unused resources of the completed projects to other projects, the easy coordination of the supply of inputs necessary for the newly-developed projects and the instant or periodic revision of efficiency.

4.3. Alternates of the goal values and interpretation of the results

The decision-making unit itself can either assign the goal values ?1, ?2 and ?3 which are found using the WGPMCDEA-CCR model or can set the goal values as the decision variable in the model. Hereby, both versions of the case were discussed consecutively:

* First Case: When goal values are set as the decision variable, the Excel Solver itself defines the values for the goals. Table 5 shows these goal values which obtain a separate value for each proj ect.

If the goal values are set as the decision variable, the goal value g1 ranges from minimum 0 to maximum 1 as it theoretically appertains to the objective of maximizing the efficiency rate whereas the goal values g2 and g3 become minimum 0 and there is no upper bound for the goal values g2 and g3. Here, the goal value g1 never becomes 1, and decisionmaking unit is already efficient because inefficiency rate will be 0 when the goal value g1 is 0. Also, as the goal value g3 appertains to the objective of the sum of deviations, it will always be greater than the goal value g2, and if the decision-making unit itself is to specify the goals, it should do so by paying attention to this. The goal values specified for each project are the minimum values, efficiency rates will not change if they get a higher value. However, if they get smaller values than the numeric values of the model, the efficiency rates will change.

* Second Case: When the values are assigned by the decision-making unit to the goal values, different alternatives can be created. Table 6 indicates some of them.

The dark colored parts in Table 6 are the efficiency rates which differ from the existing solution. Table 6 further displays what values the efficiency rates obtain when different values are assigned by the decision-making unit to the goal values ?1, ?2 and ?3. For instance, when the inefficiency rate equals 0 in alternates numbered as 8 and 12 consecutively, that is, if ?1= 0, it is observed that Project B is efficient when the values higher than the supposed values are assigned to the goal values ?2 and ?3. For a more detailed discussion, the study by Polat (2019) can be reviewed. In a nutshell, just as the goal values can be set as the decision variable, they can also be assigned by the decision-making units. This choice will depend on the orientation of the decision-making unit and efficiency rates will vary along with the goal values.

5. CONCLUSION

Based on linear programming, the DEA was an approach measuring the relative efficiency rates of the decision-making units such as businesses, organizatios, projects which used multiple inputs and multiple outputs and shared the same goals and targets. On the other hand, the GP, a multi-criteria decision-making method, was a powerful tool for finding solutions towards multiple objectives simultaneously in addition to its use for decision-making with multiple objectives, and its application was quite widespread.

In this study, the GP approach for the multi-criteria DEA model was analyzed and a case study was carried out in order to measure the efficiency of six projects developed in the automotive supplier industry with two inputs and three outputs. The problem was also solved using the GP approach for the multi-criteria DEA model and a knowledge system was proposed for a general problem structure.

Following the solution, efficient projects were noted, and the size of the decrease in inputs and the magnitude of the increase in outputs necessary to make the inefficient projects efficient were specified. The results of the proposed system and the results of the conventional DEA were compared and the divergence in the results was underlined. What type of solutions were obtained either by the assignment of goal values by the decision-making units or the allocation of the goal values as the decision variable and how the efficiency rates varied in the case of different alternatives were all together revised.

A knowledge system was devised taking into consideration the high frequency of change in the business atmosphere. As the knowledge system designed using the WGPMCDEA-CCR model was fast and functional, it was possible for the decision-making unit to make an appraisal by getting results immediately through the substitution of approximate values for unattainable goal values. Formulating a mathematical model each time was not required.

Alternative solutions could be produced by changing the goal values specified for objectives. If an additional project was added to the portfolio of the business or if there was any change in the existing projects, efficiency appraisal up to twelve projects and the efficiency comparison among the projects could be successfully undertaken. Guiding the business management in its decision-making process, this system would make it easier to reach a decision over whether to take action or not for making the goals attainable and it would further ensure the rational allotment of the resources from cancelled projects to the other projects.

A limited number of studies analyzed both the DEA and GP approaches together in the past, and different mathematical models were suggested for these studies. The WGPMCDEA-CCR model was also one of these studies. Although the WGPMCDEA-CCR model was theoretically elucidated, it was not supported with quantitative examples. With this study, the WGPMCDEA-CCR model was buttressed with a quantitative example, and a knowledge system was proposed to enhance its interactive use.

In this study, the weights of objectives were assumed to be equal; however, it was believed that it would be possible to conduct further studies by assigning a value to each weight with a prospective method afterwards. Additionally, in the following studies, this study could be enlarged to include also the undesirable output concept (quality errors, discard rates).

For all above-stated reasons, it is thought that the study will be a significant contribution to the literature, and will be useful for the ensuing studies to be undertaken subsequently.

References

REFERENCES

Bal, H., & örkçü, H. H. (2007) A Goal Programming Approach to Weight Dispersion in Data Envelopment Analysis. G.U. Journal of Science, 20(4), 117-125.

Bal, H., örkçü, H. H., & çelebio&gbreave;lu, S. (2010) Improving the Discrimination Power and Weights Dispersion in the Data Envelopment Analysis. Computers & Operations Research, 37(1), 99-107.

Banker, R., Charnes A. W., & Cooper, W. (1984) Some Models for Estimating Technical and Scale Efficiency in Data Envelopment Analysis. Management Science, 30, 1078-1092.

Boussofiane, A., Dyson, R., & Rhodes, E. (1991) Applied Data Envelopment Analysis. European Journal of Operational Research, 2(6), 1-15.

Charnes, A., Cooper, W. W., & Rhodes, E. (1978) Measuring the Efficiency of Decision Making Units. European Journal of Operational Research, 2, 429-444.

Ghasemi, M. R., Ignatius, J., & Emrouznejad, A. (2014) A Bi-Objective Weighted Model for Improving the Discrimination Power in MCDEA, European Journal of Operational Research, 233, 640-650.

Gholam, R. A., & Saeed Al-Muharrami, M. T. (2019) A Combined Goal Programming and Inverse DEA Method For Target Setting In Mergers, Expert Systems with Applications, 115, p. 412-417.

Hillier F. S., & Lierberman, G. J. (1995) Introduction to Mathematical Programming, McGraw-Hill, Singapore.

Ignizio, J. P. (1976) Goal Programming and Extensions, Lexington Books, Lexington.

Ignizio, J. P. (1985) Introduction to Linear Goal Programming, Duxbury Press.

Ignizio J. P., & Romero, C. (2003) Goal Programming, Encyclopedia of Information Systems, 2, 489-500.

Ijiri, Y. (1965) Management Goals and Accounting for Control, Rand MC Nally, Chicago.

Izadikhah, M., Roostaee, R., & Hosseinzadeh, L. F. (2014) Using Goal Programming Method To Solve Dea Problems With Value Judgments. Yugoslav Journal Of Operations Research, 24(2), 267- 282.

Lee, S. M. (1972) Goal Programming for Decision Analysis, Auerback, Philadelphia.

Polat, T. (2019) çok ölçütlü Veri Zarflama Analizi Modeli için A&gbreave;irlikli Hedef Programlama Yaklasimi ile Bilgi Sistemi önerisi ve Bir Uygulama. Dissertation (Master in Industrial Engineering with advisor Kiris, S.), Kutahya Dumlupinar University,Turkey.

Ross, A., & Droge, C. (2002) An Integrated Benchmarking Approach to Distribution Center Performance Using DEA Modeling. Journal of Operations Management, 20, 19-32.

Rubem, A. P. S., Carlos, C. B. J., Mello, S., & Meza, L. A. (2017) A Goal Programming Approach to Solve the Multiple Criteria DEA Model. European Journal of Operational Research, v. 260, p. 134-139.

Torres R. A., & Ravindran, R. (2019) Use of Interval Data Envelopment Analysis, Goal Programming and Dynamic Eco-Efficiency Assessment For Sustainable Supplier Management. Computers & Industrial Engineering, v. 131, p. 211-226.

Türko&gbreave;lu, S. P. (2017) Karar Vermede Hedef Programlama Yöntemi ve Uygulamalari, Osmaniye Korkut Ata üniversitesi Iktisadi ve Idari Bilimler Fakültesi Dergisi, 1(2), 29-

üstünda&gbreave;, E. (2009) Veri Zarflama Analizi ile Verimlili&gbreave;in De&gbreave;erlendirilmesi: çimento Sektörü ü zerine Uygulama. Dissertation (Master in Statistics), Selcuk University, Turkey.

Zografidou, E., Petridis, K., Arabatzis, G., & Dey, P. K. (2016) Optimal Design of the Renewable Energy Map of Greece Using Weighted Goal-Programming and Data Envelopment Analysis. Computers & Operations Research, 66, 313-326.