Performance Measurement in Regional Self-Government

Abstract

Regional self-government in the Republic of Croatia is regulated by law on local and regional self-government. There are 20 counties and the City of Zagreb is one of the regional self-government units. Public organisations have been implementing some paradigms for managing internal business processes, risk management, financial management and control, in last 10 years, as it was earlier implemented in private sector. In this article the authors will analyse regional self-government in the Republic of Croatia. The main goal of this article is to investigate opinions of examinees about performance measurement indicators for regional self-government using four standard perspectives of Balanced Scorecard Method (BSC). The survey, in which this article is based, in two regional self-government institutions (counties) has to determine which of BSC perspectives are the most important from the employees' point of view and to determine which performance indicators are suitable for efficiency and effectiveness in regional self-government.

Headnote

Abstract. Regional self-government in the Republic of Croatia is regulated by Law on local and regional self-government. There are 20 counties and the City of Zagreb as regional self-government units.

Public organisations have been implemented some paradigms for managing internal business processes, risk management, financial management and control, in last 10 years, as it was earlier implemented in private sector.

In this article authors will analyse regional self-government in the Republic of Croatia. The main goal of this article is to investigate opinions of examinees about performance measurement indicators for regional self-government using four standard perspectives of Balanced Scorecard Method (BSC).

The survey, on which is based this article, in two regional self-government institutions (counties) has to determine which of BSC perspectives are the most important from the employees point of view and to determine which performance indicators are suitable for efficiency and effectiveness in regional self-government.

Keywords. Regional self-government, Balanced Scorecard, Performance measurement, Performance indicators

1 Introduction

Does the organisation know the answers to the following questions: "What performance indicators should we measure?" and "What to do with those results?".

The problem statement of this research is how to determine which of the BSC perspectives is the most important and what are the performance management indicators suitable for regional self-government?

To address these issues, the BSC will be used as a source of performance indicators which will be tested by using poll survey. The survey was conducted in two regional self-government units (counties) - Varazdin County (VZC) and Koprivnica-Krizevci County (KCKZC). Analysis of Variance (ANOVA) has been used to analyse the differences between group means and their associated procedures.

The regional self-government in Croatia is restricted to deliver services to the citizens only in specific areas as follows: education, healthcare, physical and urban planning, economic development, transport and infrastructure, maintenance of public roads, planning and development of educational, social and cultural institutions, issuing of construction and zoning permits, and other documents regulated by special laws [1].

2 Balanced Scorecard

The Balanced Scorecard term can be traced back to 1990 when case studies about innovative performance -measurement systems were examined by David Norton and Robert Kaplan and representatives of different companies from different industries [3]. The group discussions led to an expansion of the scorecard and Kaplan and Norton labelled it as "Balanced Scorecard"[3]. The findings were summarized in an article "The Balanced Scorecard - Measures that Drive Performance" and published in "Harvard Business Review" (January - February 1992).

Legal regulation for public institutions in the Republic of Croatia recognizes BSC perspectives and regulated them by various laws. BSC is useful method for benchmarking, measuring, management and strategic management.

The basic idea of BSC concept is to extend number of fields for measuring and controlling parameters from financial with other three perspectives, recommended by BSC, for observing the business system. This concept gives possibility to understand the correlations between business perspectives, the importance of each of them and to manage organizations in complex environments [3].

Secondly, number and type of perspectives depends on type of organization and the usual ones are [1]:

* Financial perspective means control of usual financial parameters of business, and the main restriction is that it has to be observed by the shareholders and stakeholders' point of view.

* Customer perspective - means that management has to define how to create value for our customers? Basically this perspective has to assure quality products and services, effective delivery and customer satisfaction.

* Internal perspective (or Internal business process) - basically is focused at which processes must we excel to satisfy customers and shareholders. Special properties of business system are organizational effectiveness, productivity, turnover cycle and costs.

* Learning and growth (or Innovation and Learning) has to be focussed to continual learning and improvements.

Figure 1 shows the BSC with four perspectives that are connected to the vision and strategy, to measurable objectives with defined initiatives and activities to direct organization towards business objectives [3].

Kaplan and Norton [3] have emphasized the importance to adjust the BSC to organizational context.

Robert Kaplan has explained that the BSC was originally developed for the private sector to overcome deficiencies in the financial accounting model, which fails to signal changes in the company's economic value as an organization makes substantial investments (or depletes past investments) in intangible assets like skills, motivation, and capabilities of its employees, customer acquisition and retention, innovative products and services, and information technology [3].

Brumec has developed [1] guidelines for objective and measures assessment in non-profit and for-profit organizations presented in Table 1.

In the literature the Financial perspective and Customer perspective are Primary objectives or results to be achieved but secondary objectives or determinants of success to achieve are Internal Business Processes and Innovation and Learning [4].

In these days non-profit organizations encounters an increasing competition for funding, therefore the accountability and performance management has become urgent topic [2]. They have to pay more and more attention to measure the results of success and how effectively and efficiently they meet the needs of their constituencies [2].

3 The problem statement

The aim of this research is to verify by poll survey what BSC performance measurement indicators can be used in regional self-government, and which of the BSC perspective is the most important.

4 The research methodology

A questionnaire was created by using brainstorming method to find opinions and attitudes about performance measurement indicators from the examinees point of view.

The research was conducted in KCKZC and VZC during the March 2015 using a Google Forms [7] Internet questioner.

The questionnaire has 39 questions distributed in 5 categories shown in Table 2. The Likert type questions were used in most cases and they are represented in Table 1. without brackets. The examinees should answer the question by putting a bullet mark under the one of suggested answers (1-5), where the mark 1 meant "I completely disagree" and mark 5 meant "I Completely Agree". Other types of questions were mostly used for general data and demographic data.

4.1 The target examines and procedure

The target examines were all employed in VZC and KZKZC. It was not obligatory for employees to answer the questions, but those who decided to give the answers have to give the answer to all questions, anonymous. The total number of employees, which were employed on the February 28th 2015 were 99 employees in VZC and 79 employees in KCKZC collected from departments responsible for human resources in both counties.

The questionnaire was distributed to examinees by e-mail as a link in both counties. There were 178 questionnaires distributed and 104 were completed, e.g. 58%. VZC employees have completed 62 questionnaires (63%) and KZKZC employees have completed 42 or 53%. The qualitative and quantitative controls were performed on collected data to identify incompatible factors.

4.2 The reliability of measurement instrument

To ensure that the repeated measuring would show the equal measurement indicator, it is necessary to measure the reliability of measurement instrument. The satisfying reliability is considered if coefficient of reliability is 0.70, some authors use 0.75 or 0.80 as cutoff value, while others are as lenient as 0.60. In general this varies by discipline [5]. Cronbach alpha test is mostly used to apply the reliability estimation by using internal consistency coefficient.

5 Research results

Prior to the data analysis, the reliability and validity of the measurement instrument are explored and it is presented in previous chapter. To ensure the validity, the construction of the instrument content and questions were grouped regarding BSC perspectives. Questions were selected in order to point out some performance indicators. The reliability is verified using Cronbach alpha coefficient. In Table 3. the Cronbach alpha coefficient is presented for every perspective. The coefficient is higher than 0.70 for all perspectives and so it can be concluded that the instrument is reliable. The average mark (AVG Mark) for every perspective and overall has been calculated and presented in Table 3.

A demographic profile analysis reveals that 66% of overall respondents are female and 34% are male. Surprisingly, a majority of the respondents are between 31-50 years old. Regarding academic qualification, majority of respondents 81% (84) have higher education than secondary school. The precise results are represented in Table 4.

Among the 39 questions in questionnaire, one was appointed, "How would you rank the importance of listed BSC perspectives for organizational strategic management?" The rank scale was 1 - 4, where 1 implicates the smallest priority and 4 implies highest priority. This question is important because it reveals if the examines do understand their role in the society.

The Customer satisfaction perspective has been ranked as most important; the 97 examinees (93%) ranked it with highest grade (4). That implicates the good understanding of role and mission of regional self-government.

The Financial perspective is second important with 89 examinees which ranked it with highest grade. The split opinion between counties was with other two perspectives were slightly majority of VZC examinees gave the lower rank 3 for perspectives Internal business process (42 - rank 3, 34 rank 4) and Innovation and Learning (42 - rank 3, 40 rank 4). KCKZC examinees gave the rank 4 for those perspectives.

Third most important perspective is Innovation and Learning and fourth perspective is Internal business process.

Descriptive statistics is used for data analysis. To identify the area where the organizations have better or weaker performance the percentages for all items were ranked.

Table 5. represents the statistical summary of responses ranked by overall average mark. The performance indicators were ranked in quartiles, the green colour labels first quartile indicators and red colour labels fourth quartile indicators.

In the first quartile regarding performance indicators ranking, 3 performance indicators are from Customer Satisfaction perspective (Q25, Q29, Q30) and three from Internal Business Process perspective (Q12, Q14, Q11). Therefore it can be concluded that performance indicator ranking is in accordance with perspective importance ranking. Very similar marks in the first quartile were given from both counties with slightly difference in two indicator ranking.

Fourth quartile shows similar distribution of ranks, with some differences in ranking two indicators.

The most appropriate method to evaluate the difference among the collected data between BSC perspectives in VZC and KCKZC counties is the Analysis of variance. The analysis of variance is a set of analytic procedures based on a comparison of two estimates of variance [6]. One estimates the differences among scores within each group, and the second difference between group means, and this is considered to be a reflection of group differences or a treatment of effects and errors. If these two estimations of variance do not differ significantly, the conclusion is that all the groups means come from the same sampling distribution of means, and that the slight difference among them is due to a random error. Also, if the group means differ more than expected, it is concluded that they come from different sampling distributions of means and the null hypothesis should be rejected [6]. Accordingly, the analysis of variance (ANOVA) is used for a:

* Testing the hypothesis on the equality of arithmetic means of the k basic groups, by using the independent random samples,

* The analysis of the drafts of statistic experiments, and

* Testing the hypothesis on parameters (variables) in regressive models.

The analysis of variance according to the number of independent variables recognizes the univariate and multivariate analysis of variance.

For the purpose of this research for testing hypothesis on means equality of the two basic groups, in the analysis of variance were used:

* The null hypothesis (H0) - arithmetic means of all basic groups are equal, meaning the differences among the arithmetic means can be described as random and are not significant, and

* The alternative (H1) hypothesis - the arithmetic means are not equal, meaning the differences among the arithmetic means cannot be described as random

Prior to analysis of variance method, following assumptions should be examined:

1. The variable whose arithmetic mean is tested is distributed according to the normal distribution. This assumption will be tested using the Kolmogorov-Smirnov test. With the probability value p and significant level α=0.05, it can be concluded that all distributions are close to normal distribution.

2. The distributions of basic groups have equal variances. This assumption can be tested by the Leven test and the Brown-Forsythe test. The Leven test is mostly used to test the samples of equal sizes. While testing the samples of different sizes the Brown-Forsythe test is more sensitive and robust. If variances are not equal, the analysis of variance is pursued by the weighted analysis of variance, i.e. the Welch analysis. The homogeneity of variances was tested using Leven and Brown-Forsythe tests as shown in Table 6.

3. The samples chosen from the basic groups are independent. The independence of samples is ensured by randomly chosen participants.

All assumptions for Analysis of variance are satisfied with results of previously conducted tests.

The second part of Table 6 shows the results of Variance analysis for identifying the differences between arithmetic means of samples from VZC and KCKZC.

* (H0) - the arithmetic means of all basic groups are equal,

* (H1) - the arithmetic means are not equal

* Significant value α =0.05

Conclusion will be based on the probability level - p and it will be used for all perspectives. With the significant value α =0.05 and the p value higher than 0.05, the H0 hypothesis cannot be rejected (H0 should be accepted), meaning that differences between samples arithmetic means are random and not significant. Contrary, if the p value is less than 0.05, H1 hypothesis can be accepted, meaning that the differences between samples arithmetic means are significant.

Regarding this research p - values for all perspectives are higher than 0.05 and the H0 hypothesis is accepted. That means, for perspectives Customer satisfaction, Financial Performance, Internal Business Process and Innovation and Learning the differences between samples arithmetic means are not significant. In other words, assumption based on given results is that there is no significant difference in applying BSC perspectives between the VZC and KCKZC.

Strong correlation (Figure 2.) has been discovered between Age of examines and following BSC perspectives: Internal business process Perspective (0.92); Innovation and learning (0.92); Customer Satisfaction (0.98); and Financial performance (0.99).

Positive correlation has been discovered between Innovation and learning and Customer Satisfaction perspective (0.88) and Financial performance perspective (0.86) and strong correlation between Customer Satisfaction perspective and Financial Performance perspective (0.97).

Regarding Education (Figure 3.), Internal business process Perspective has strong correlation with Customer Satisfaction (0.95), positive correlation with Innovation and learning (0.75). Innovation and learning has positive correlation with Customer Satisfaction perspective (0.77) and but there is no correlation with Financial Performance perspective (0.04).

Considering Work experience (Figure 4.), strong correlations have Internal Business Process perspective with all 3 other perspectives (Innovation and Learning 0.84, Customer Satisfaction 0.90 and Financial Performance 0.97). Innovation and learning perspective has strong correlation with Customer Satisfaction perspective (0.94) and Financial performance (0.88). Customer Satisfaction and Financial perspective has strong positive correlation (0.87).

When the Duties are in focus (Figure 5), there have been discovered only positive correlations between perspectives. Very strong correlations are between Internal business process perspective and Customer Satisfaction (0.95) and strong with Innovation and Learning (0.89) and Financial Performance (0.74). Innovation and learning has strong correlation with Customer Satisfaction perspective (0.95) and positive correlation with Financial performance (0.76). Customer Satisfaction has positive correlation with Financial performance perspective (0.75).

6 Conclusion

The limited amount of resources leads organisations to measure and to manage the important performances. Measuring of the organizational performance helps to recognize the leader in the area of competence, to compare with the leader and to identify the scope of possible improvements.

It is possible to use BSC effectively in non-profit organizations to measure the performance indicators and also use them for benchmarking. It means to align business with vision and strategy and to benchmark with other organizations, therefore the performance measurement indicators from BSC are used in this research. Authors have intention to define the measures in regional self-government using BSC perspectives with performance indicators. The differences between group means and their associated procedures were analysed by using Analysis of Variance (ANOVA). The 93% respondents from both counties ranked the Customer satisfaction as the most important and this fact implicates the high awareness of employees about the role and mission of regional self-government.

The results analysis of the research pointed out surprisingly high level of knowledge about performance measurement, BSC perspectives. Some suggestions for performance measurement indicators were given too. The average marks in every perspective were very similar for both counties, and even the marks of every indicator were similar. When indicators grouped in quartiles, some minor differences showed up.

The variance analysis proves that there is no significant difference in applying BSC perspectives between the VZC and KCKZC.

The results of this survey indicate that importance of measuring performance indicators is recognized in regional self-governments of this particular area of Croatia and it is implemented.

In this article authors have presented possibilities to identify the area where the organizations have better or weaker performance (or similar) which is vital for further possible research of implementing benchmarking in regional self-government.

It is also important to stress some constraints, such as that research has covered only 2 (north Croatian counties) of 21 regional self-government units and that minor number of employees work there compared to overall number of employees in regional selfgovernments. The great opportunity is to implement the methodology and expand the research to other counties in different regions or even abroad.

References

References

[1] Brumec, J., Furjan, M. T.: "Design Methods for Measures in Balanced Scorecard," presented at the CASE 18 Methods and Tools for Information and Business Systems development, 2006.

[2] Kaplan, R. S.: "Strategic Performance Measurement and Management in Nonprofit Organizations," Apr. 2001.

[3] Kaplan, R. S., Norton, D. P..: The Balanced Scorecard: Translating Strategy into Action. Boston, Massachusetts: Harward Business School Press, 1996Kloot, L. and Martin, J.: "Strategic performance management: A balanced approach to performance management issues in local government," Manag. Account. Res., vol. 11, no. 2, pp. 231-251, Lipanj 2000.

[4] L. Kloot, L., Martin, J.: "Strategic performance management: A balanced approach to performance management issues in local government," Manag. Account. Res., vol. 11, no. 2, pp. 231-251, Lipanj 2000.

[5] Nardo, M.: "Handbook on constructing composite indicators: Methodology and user Guide." OECD Statistics, 2008.

[6] Tabachnick, B. G.: Using Multivariate Statistics. New York: Harper Collins College Publishers, 2006.

[7] "Google Forms - Easy-to-create surveys and forms for everyone." [Online]. Available: www.google.com/intl/en_uk/work/apps/business/ products/forms/. [Accessed: 14-Apr-2015].

Law in the Republic of Croatia

..., Zakon o lokalnoj i podrucnoj (regionalnoj) samoupravi, "Narodne novine" 33/01, 60/01, 129/05, 109/07, 125/08, 36/09, 150/11, 144/12 i 19/13 - procisceni tekst.

..., Zakon o sustavu unutarnjih financijskih kontrola u javnom sektoru, "Narodne novine" br. 141/06.

..., Zakon o reviziji, "Narodne novine" br. 146/05, 139/08, 144/12, 19-Dec-2012.

..., Zakon o proracunu, "Narodne novine", br. 87/08, 136/12, 15/15.

.., Zakon o fiskalnoj odgovornosti, "Narodne novine", br. 139/10, 19/14