RELATIVE EFFICIENCY OF UNIVERSITY LIBRARIES IN CROATIA: A WINDOW DATA ENVELOPMENT ANALYSIS APPROACH

Abstract

The main objective of this paper is to assess the efficiency of university libraries in Croatia in the period from 2018 to 2022, taking into account both the years before and after COVID-19. For this purpose, the study employed the well-known non-parametric Data Envelopment Analysis (DEA) method, specifically the window DEA (WDEA), to analyse a sample of 8 university libraries in Croatia. The study considered three inputs (Number of books held, Number of library staff, and Actual allocated budget) and three outputs (Circulation, Number of books added, and Number of registered members). The findings indicate that none of the university libraries in Croatia demonstrated relative efficiency during the observed period. These results provide valuable insights for university management and policy makers. This study is characterised by its original approach, as it is the first known attempt to use the DEA method to assess the performance of academic libraries in Croatia and the South East European region.

Alternate abstract:

Glavni cilj ovog rada je procijeniti ucinkovitost sveutilisnih knjiznica и Hrvatskoj и razdoblju od 2018. do 2022. godine, uzimajuci и obzir godine prije 1 nakon pandemije COVID-19. U tu svrhu, istrazivanje koristi poznatu neparametrijsku metodu Data Envelopment Analysis (DEA), tocnije njezinu varijantu window DEA (WDEA), kako bi analiziralo uzorak od osam sveucilisnih knjiznica и Hrvatskoj. Uistrazivanju su koristena tri ulazna pokazatelja (broj knjiga и knjiznici, broj zaposlenika knjiznice i stvarno dodijeljeni proracun) te tri izlazna pokazatelja (broj posudbi, broj novih knjiga i broj registriranih lanova). Rezultati pokazuju da nijedna od analiziranih sveucilisnih knjiZnica nije pokazala relativnu ucinkovitost tijekom promatranog razdoblja. Ovi nalazi pruzaju vrijedne uvide za upravljanje sveucilistima i donositelje politika и podruéju visokog obrazovanja. Ovaj rad je jedinstven po svom pristupu, buduci da predstavlja prvi poznati pokusaj primjene DEA metode za procjenu uspjesnosti akademskih knjiznica и Hrvatskoj 1 regiji Jugoistocne Europe.

Headnote

Received: 13. 11. 2024.

Accepted: 30. 6. 2024.

ABSTRACT

The main objective of this paper is to assess the efficiency of university libraries in Croatia in the period from 2018 to 2022, taking into account both the years before and after COVID-19. For this purpose, the study employed the well-known non-parametric Data Envelopment Analysis (DEA) method, specifically the window DEA (WDEA), to analyse a sample of 8 university libraries in Croatia. The study considered three inputs (Number of books held, Number of library staff, and Actual allocated budget) and three outputs (Circulation, Number of books added, and Number of registered members). The findings indicate that none of the university libraries in Croatia demonstrated relative efficiency during the observed period. These results provide valuable insights for university management and policy makers. This study is characterised by its original approach, as it is the first known attempt to use the DEA method to assess the performance of academic libraries in Croatia and the South East European region.

KEYWORDS: relative efficiency, university library, Croatia, data envelopment analysis, window DEA.

SAZETAK

Glavni cilj ovog rada je procijeniti ucinkovitost sveutilisnih knjiznica и Hrvatskoj и razdoblju od 2018. do 2022. godine, uzimajuci и obzir godine prije 1 nakon pandemije COVID-19. U tu svrhu, istrazivanje koristi poznatu neparametrijsku metodu Data Envelopment Analysis (DEA), tocnije njezinu varijantu window DEA (WDEA), kako bi analiziralo uzorak od osam sveucilisnih knjiznica и Hrvatskoj.

Uistrazivanju su koristena tri ulazna pokazatelja (broj knjiga и knjiznici, broj zaposlenika knjiznice i stvarno dodijeljeni proracun) te tri izlazna pokazatelja (broj posudbi, broj novih knjiga i broj registriranih lanova). Rezultati pokazuju da nijedna od analiziranih sveucilisnih knjiZnica nije pokazala relativnu ucinkovitost tijekom promatranog razdoblja. Ovi nalazi pruzaju vrijedne uvide za upravljanje sveucilistima i donositelje politika и podruéju visokog obrazovanja. Ovaj rad je jedinstven po svom pristupu, buduci da predstavlja prvi poznati pokusaj primjene DEA metode za procjenu uspjesnosti akademskih knjiznica и Hrvatskoj 1 regiji Jugoistocne Europe.

KLJUENE RIJECI: relativna ucinkovitost, sveuciliSna knjiznica, Hrvatska, data envelopment analiza, window DEA.

(ProQuest: ... denotes formulae omitted.)

1. INTRODUCTION

University libraries serve as central hubs of learning, information literacy, and research, and are the cornerstone of all educational institutions. Higher education and university libraries are committed to fostering the information and communication skills of their users, supporting not only the curricula and scholarly projects of universities, but also the broader scholarly ecosystem and the research and creative culture of the entire academic community. Based on the University Library Standard 2022 (Standard za visokoSkolske knjiZnice), the library's services are primarily aimed at students, teachers, and professional associates at the university. However, they are also available to other members of the wider social community who are academically and professionally active. Libraries are essential in modern educational and scientific systems, and it is imperative that they continually integrate modern advances in librarianship into their operations. However, libraries have faced questions of necessity throughout history, and the "culture of change and continuous development in the context of libraries is nothing new" (Bawden & Robinson, 2015). Libraries are expected to provide the best possible services (Najafi, 2020), which is a major challenge in the information age. Therefore, it is fair to say that university libraries face a variety of challenges, including budget cuts, declining user numbers, and the task of providing high-quality resources to their customers. A university library can be understood as "an institution where the professional staff establishes the operational framework for converting measurable resources (inputs) into student learning and faculty research (outputs)." (Chen, 1997). A recent study by Lynch (2017) found that academic libraries face several challenges, including limited budgets, the rapid pace of technological advancement, and changing user expectations. However, the study also suggests that these challenges provide libraries with the opportunity to innovate and adapt, ultimately strengthening their relevance in the academic community. In our ongoing research, we will look at the process of evaluating the effectiveness and quality of university libraries. Library service assessment is usually about meeting users' expectations and utilising resources effectively to provide quality services (Najafi, 2020). Library management assessment has a long history, with the first contributions dating back to Morse's 1968 "Library Effectiveness" (Shahwan & Kaba, 2013). Subsequent studies have further advanced the understanding and evaluation of library performance (Shahwan & Kaba, 2013). Tavares (2018) emphasises that the information age has brought challenges in managing collections and ensuring their accessibility. The shift to electronic resources has necessitated the development of new management approaches. As a result, university libraries are under pressure to utilise their resources effectively to deliver services and assess their impact on users (Tavares, 2018; Shim, 2003).

In 2022, there were a total of 102 higher education, university, and scientific libraries in the Republic of Croatia. The distribution includes 91 higher education libraries, 9 university libraries, and 2 scientific libraries. The legal regulation under which Croatian higher education libraries operate is the Law on Libraries (Law on Libraries and Library Activities (NN 17/19, 98/19) and the Standard for Higher Education Libraries.

The COVID-19 pandemic brought many challenges to libraries. The extent of the consequences is now visible, as the number of library users has not returned to pre-pandemic levels and users' habits have changed (De Groote, 2021). Adapting to the new situation is a challenge that has always accompanied libraries, and libraries must adapt to new situations. During COVID-19, the number of library users decreased drastically and users turned to other types of library services (Wakeling, 2022). During the COV1D-19 pandemic, there was a notable increase in the use of virtual support and communication with customers. The national crisis headquarters consistently issued daily public information on the implementation of the increasingly strict epidemic control measures (Lauri Korajlija & Jokic-Begic, 2020).

At its meeting on 13 March 2020, the Croatian government adopted the Decision on the suspension of physical classes in all educational institutions throughout the country, including universities, secondary and primary schools, and the regular work of pre-school educational institutions as part of the new concept of distance learning. This decision also suspended the work of university libraries. Just one month later, on 23 April 2020, the work of the libraries was made possible again on the basis of the Government's decision on measures to resume economic and other activities after the declared epidemic of the COVID-19 disease.

On this occasion, the National and University Library in Zagreb (NSK) drew up working guidelines that applied to all libraries in Croatia. Libraries throughout the Republic of Croatia began working with users on 27 April 2020, in compliance with all anti-epidemiological measures and strict social distancing (Ministry of Culture and Media, 2020). The guidelines for working with users and staff included maintaining distance and a limited number of users and staff in the library's workspaces to reduce physical contact, keeping a prescribed distance of up to 2 metres and ensuring compliance with the highest hygiene standards. The recommendations also included several technical guidelines in the form of a separate entrance/exit to the library and a hand sanitiser dispenser at the entrance/exit. Librarians were required to wear protective gloves and a mask covering the nose and mouth. It was also recommended that a protective partition be installed at the desk to physically separate the employee from the user. When returning the book material, the user would place the book in a marked place (box, basket) on the counter. The returned book, which the user had placed in the basket/box, was taken to a previously prepared area of the library - a separate room for disinfection and quarantine. The books were quarantined for 72 hours and then returned to the storage location.

Online learning and hybrid teaching models have become the "new normal" for the academic community (Martzoukou, 2021). Distance learning has proven to be a highly effective solution for academic institutions, especially in containing and preventing the spread of the coronavirus while ensuring uninterrupted continuation of the learning process (El Refae, Kaba, & Eletter 2021). In the period before the COVID-19 pandemic, universities had already in- tegrated a wide range of new technologies, online tools and virtual resources into their teaching, academic and research activities. The outbreak of the pandemic accelerated and intensified the adoption of digital resources across the academic landscape (Valverde-Berrocoso, 2021). The main objective of this research is to assess the relative efficiency of university libraries in the Republic of Croatia. To achieve this, the input-oriented approach of DEA with three inputs and three outputs was used. DEA is a non-parametric method that enables an assessment of the relative efficiency and performance of decision-making units (DMUs) which use multiple input variables to generate output variables. However, when the sample at hand is too small for the chosen multiple inputs and outputs (as in this case), the WDEA is most appropriate as it allows for an increase in the number of DMUs in the sample (Cvetkoska et al, 2020). Five years of panel data (from 2018 to 2022) are used in this study.

The rest of the article is organised as follows. In the next section a literature review is given. Section 2 presents the methodology and data. Thereafter, section 3 presents the results and findings. Furthermore, a discussion of the main findings is opened and the most interesting results are addressed and analysed. The final, fourth section concludes the paper, addresses its limitations and provides directions for future work.

2. LITERATURE REVIEW

For this literature review, we analysed the Web of Science and Scopus databases. The aim was to search the two most reputable and well-known citation databases and find papers that deal with the efficiency of university libraries using this particular method. The search strategy was to search for the keywords "DEA (i.e. "Data Envelopment Analysis") and "University Library". After a comprehensive analysis and review of the sources on the topic, 94 articles in Scopus and 50 in Web of Science were screened after the initial selection. The specific criteria were determined based on the field of science and the limitation of English language usage in the publication. This included disciplines such as Information Science and Economics. The papers listed in Table 1 were analysed, focusing exclusively on the efficiency of university libraries using the DEA method. Based on an analysis of the available sources using the DEA method, it was found that most research on libraries focuses on variables related to traditional library parameters.

A plethora of studies have been published on the topic of academic library efficiency and performance in recent decades. Tavares et al. (2018) found that Asian countries are actively involved in research on university library efficiency assessment. Taiwan leads the way with the highest number of published articles on this topic. Specifically, nine articles were published in this context. The studies applying the DEA method are the focus of this research section.

In this paper, DEA is used to assess the relative efficiency of university libraries in Croatia in the period from 2018 to 2022, i.e. it is an analysis of 8 public university libraries in Croatia that are active in the entire observation period 2018-2022, and special attention is paid to the application of the WDEA technique.

In 1997, Chen conducted a study using a DEA model to examine the overall efficiency as well as the technical efficiency and scale efficiency of 23 university and college libraries in Taipei City and County in 1995 - the first article to focus on the evaluation of library performance using DEA. The results showed that 11 university libraries were highly efficient and 9 of them exhibited credible academic performance function. In contrast, only two university libraries were identified with weaker research capabilities. In a study conducted by Kao & Lin (1999), 24 university libraries in Taiwan were evaluated using the DEA model based on Pareto optimality. The evaluation criteria included two aspects - overall measure and ratio measure. The overall measure evaluated "the total resources and services" provided by the library, while the ratio measure took into account the size of the university and focused on the resources and services shared by students and faculty. Kao & Lin (2004) included the same 24 university libraries in their empirical study to evaluate them based on the overall measure and the ratio measure. The results showed that only four libraries provided adequate resources and services to their clients. In 2006, Reichmann & Sommersguter-Reichmann conducted a study on the technical efficiency of 118 academic libraries in Canada, Germany, Austria, Switzerland, Australia, and the United States. The study provides insights into the technical efficiency of libraries on both an individual and aggregate level and compares the European group of libraries with the non-European group. The results show that almost a third of university libraries are technically efficient. The study also found that the non-European best practise line outperforms the European best practise line.

In Table 1, a comprehensive overview of the periods, selected variables, samples, utilized DEA models, and findings of recent relevant literature is presented. The most recent bibliometric analysis of library performance evaluation using the DEA methodology was published by Najafi et al. (2020), in which the authors analysed the academic literature including the scientific databases Scopus and Web of Science. For the core study, the authors selected 44 articles published between 1997 and 2018. According to the study, most of the studies were conducted by experts from the fields of economics, management, and engineering. However, experts from the field of library and information science rarely conduct these studies. Interestingly, a significant number of studies were conducted in Taiwan and the United States, with a focus on evaluating the performance of university libraries. The analysis of the papers showed that different models of DEA techniques were used in the collected articles. As many as 41% of the published articles used the basic CCR, BCC or a combination of two models (CCR & BCC) (Najafi et al 2020). After a thorough search of various databases, including those specifically related to Croatia, it was found that there is no existing work on this topic in Croatia.

The literature review shows that there is no clear consensus on the appropriate input and output variables to measure the efficiency and performance of libraries. The research covers different countries and variables, providing different findings that are representative of a wide range of cultural, social, and economic contexts. In examining these studies, it is important to consider the unique contextual factors of each individual library.

3. METHODOLOGY AND DATA

This study analyses the relative efficiency of university libraries in the Republic of Croatia by applying the WDEA technique for the period 2018-2022.

DEA, originally introduced to the Operations Research (OR) literature in 1978 (Pjevéevié et al, 2012), is a popular linear programming technique and currently the leading and widely used non-parametric method used to measure the efficiency of organisations referred to as homogeneous decision-making units (i.e. units that use the same inputs and the same outputs) compared to the efficiency of other organisations (DMUs) operating in the same industry. DMUs can be banks, insurance companies, hospitals, universities, cities, organisations, etc. The basic DEA models (CCR and BCC) were introduced more than four decades ago (CCR in 1978 and BCC in 1984) and were originally developed to assess the performance and relative efficiency of the non-profit sector. However, after its application in many industries, the basic CCR DEA model has been extended to several hybrid versions over the years, and the WDEA is one of its best-known versions (AlKhars et al., 2022).

The CCR DEA model (used in this study) can be transformed into a mathematical linear model in this way:

...

The WDEA method was originally developed by Klopp (1985), who introduced this DEA approach while serving as "a chief statistician for the U.S. Army Recruiting Command" (Cooper et al, 2007). It is "a popular, effective, and applicable method for dynamic performance assessment of peer decision-making units (DMUs) (Peykani et al, 2021). Its basic concept lies in "subdividing the entire analysed period of data into smaller windows to be analysed separately" (Paradi et al, 2018). Therefore, the WDEA is often recommended for analysing efficiency trends and their fluctuations over time and is used "to verify productivity change over time" (Repkova, 2014). This method adds a dynamic perspective to the DEA methodology (Fotova Cikovié et al, 2022). However, the variations in the efficiency of the various DMUs over time can only be analysed "given the assumption that the production possibilities remained the same during the whole period of analysis" (Cvetkoska & Fotova Cikovié, 2020). The WDEA, working on the principle of moving averages, offers great advantages as it treats each DMU in different periods (i.e. year) as a different DMU/organisation, thus providing a more comprehensive and multi-layered overview of each DMU's performance over time. One of the bestknown advantages is the ability to increase the number of DMUs observed when the number of DMUs in the sample is limited. This was the case in this study. Therefore, due to the small sample size (8 university libraries), the decision was made to conduct the WDEA. The relationship between the number of different organisations, the width of the window used, the number of windows in the DEA model and the number of periods analysed can be calculated using the following formula:

...

Number of different organizations=n-p-w

Where:

w = the number of windows,

k = the number of periods,

p = width of the windows,

n = the number of organizations.

The selected window width is also an important issue in the WDEA technique. Namely, it "should be as small as possible to reduce unfair comparisons over time but, at the same time, should be large enough to generate a sufficient sample size" (AlKhars et al, 2022). Assuming there are DMUs (DMU,j=1..,n) that consume inputs (x;i=1..,m) to produce s outputs (V,;r=1...5), observed in periods. Let represent an observation in period having an input vector

...

and an output vector.

...

Furthermore, consider a window that starts at time | (1<1< Т) with a window width 1<t-1.>

...

Substituting the inputs and outputs of into a model, the efficiency results of each DMU in the WDEA model can be calculated.

The literature review shows that there is no clear consensus on the appropriate input and output variables for evaluating the efficiency and performance of libraries. The selection of input and output variables for the DEA model is a very important step that can have a significant impact on the efficiency results. This topic has been addressed in the academic literature by Toloo & Tichy (2015) and Galagedera & Silvapulle (2003), who claim that only the selection of the correct and appropriate input and output variables and performance measures for the DEA model can ensure accurate and meaningful efficiency results. Since the selection of inputs and outputs is an important part of the DEA model, the selection of variables is based on Mohamed Shahwan & Kaba (2013) as shown in Table 2.

The data were collected manually from the official annual reports of the individual university libraries included in the sample. The sample consists of 8 university libraries in Croatia that were active throughout the observation period 2018-2022. For this reason, the library of the University of Slavonski Brod was excluded, as it only started operating in 2020. In this study, the input-oriented WDEA method was applied to the sample of eight university libraries that were active throughout the analysed period of five years (2018-2022). A window of 3 years was also selected. The length of the window was determined using the formula of AlKhars et al. (2022), as shown in Table 3.

Each window covers 3 years (e.g. window 1 covers 3 years: 2018, 2019 and 2020; while the next window (window 2) omits the data for 2018 and adds the data for 2021), and the last window (window 3) starts with the year 2020 and ends with 2022, as shown below in Table 4. There are 32 university libraries in each window, and the number of "different" university libraries is 72.

The DEA-Solver-LV software was employed for solving the specified Window-1-C DEA model. Further information on this software can be found in Cooper et al. (2007). The efficiency scores for each library can be used to determine which university libraries are relatively efficient and which are not. Efficient libraries are assigned an efficiency score of 1, i.e. 100%, while inefficient libraries are assigned an efficiency score of less than 1 (below 100%).

4. RESULTS AND DISCUSSION

This paper measures the relative efficiency of eight university libraries in Croatia using the WDEA from 2018 to 2022. A balanced data panel was used for the sample. The results of the input-oriented CCR DEA window model, shown in Table 3, show that there were many fluctuations in relative efficiency during the analysed period. The Line Sparkline column shows the trend graphically. Looking at the average efficiency results for the entire period analysed, the university libraries in Split, Rijeka and Dubrovnik have the highest average efficiency (0.946; 0.903 and 0.894 respectively), while the university libraries of the University of North and the University JJ Strossmayera in Osijek have the lowest relative efficiency (0.443 and 0.586 respectively). These libraries and the university Library of the University of Zagreb did not achieve a relative efficiency of 100% in any of the years analysed. The university libraries of the University of Dubrovnik and the University of Rijeka, on the other hand, achieved an efficiency of 100% in three of the five years analysed.

Interestingly, when analysing the results for the entire public sector, the average relative efficiency values show that the first COVID-19 and post-COVID-19 years (2020 and 2021) were the most successful (85.66% in 2020 and 83.54% in 2021), while the university libraries in Croatia had the lowest efficiency in 2018 with 69.32%. This contradicts the assumption that the COVID-19 pandemic has had a severe impact on this public sector and requires further research that would include more variables and a longer timeframe.

Due to the larger fluctuations in their results, three of the university libraries from the sample (the University of Juraj Dobrila in Pula, the University of Rijeka and the University of North) have piqued our interest, so we will analyse and elaborate on their ef- ficiency results in more detail. Figure 1 shows graphically the fluctuations in their efficiency per year. Thus, the library of the University of Juraj Dobrila in Pula recorded an efficiency of 100% in 2018, followed by a decrease in 2019 and 2020 to 82.62%, up to the full relative efficiency of 100% in 2021, followed by a very sharp decrease to 65.23% in 2022. The library of the University of Rijeka started the analysis with a relatively low efficiency (55.24% in 2018), experienced an increase to 96.27% in 2019 and thus remained at the maximum relative efficiency of 100% in 2020, 2021 and 2022. The university Library of the University of North had low relative efficiency values in all observed years, except for the sudden increase in the COVID-19 year 2020 to 77.61%, followed by a sharp drop in efficiency in 2021 and 2022 (35.66% and 37.61% respectively). The main reasons for these efficiency values lie in the size of the university Library and the newness of the University North in general. However, these results require additional research, which is planned for further work.

This is the first empirical study to analyse and evaluate the efficiency of university libraries in Croatia. Consequently, this study is a stepping stone for future work in this area and for the increasing application of DEA in public sector evaluation worldwide. It can be seen as an asset for the academic community, but also provides invaluable insights and metrics for university library management (as well as university management), regulatory bodies, the local community, the interested public and other stakeholders. However, the main contribution of the conducted research lies in the generalisation and specialisation of the results (i.e. the entire public sector of university libraries in Croatia was examined as a whole, but also each individual university library).

5. CONCLUSION

During the COVID-19 pandemic, university libraries have once again proven that they can rise to the challenges and embrace change. Libraries are expected to help shape change in the communities in which they operate, and the COVID-19 pandemic has changed the traditional role of libraries. Libraries are the centre of the community in which they operate, and academic libraries are the heart and essence of bridging the informational, digital and social divide.

The academic community is a place where such services and programmes can be developed, and university libraries should be central places for e-learning and e-information sharing. The 'new normal' provided a change of focus and accelerated the changes that were necessary and inevitable.

In this study, the WDEA is used to assess the relative efficiency of university libraries in Croatia. The WDEA method with three input and three output variables was applied to a sample of eight university libraries. The observed period covered two years before the COVID-19 pandemic and two years after the pandemic (i.e. from 2018 to 2022).

Based on the results of the DEA Window I-C model, the average efficiency of the Croatian public sector of university libraries is 79.44%. None of the eight university libraries analysed was relatively efficient during the entire study period, so there is much room for improvement in their performance. Three of the most efficient university libraries are those of the University of Split, the University of Rijeka and the University of Dubrovnik (with average efficiency scores of 0.946, 0.903 and 0.894 respectively). The least efficient university libraries, on the other hand, are those of the University of North (with an average efficiency score of 0.443) and the University of JJ Strossmayer in Osijek (with an average efficiency score of 0.586). The university libraries of the University of Pula, the University of Zadar and the University of Zagreb are slightly behind with scores of around 85% (i.e. 0.894, 0.850 and 0.864 respectively). In addition, the inconsistent and quite interesting results of four university libraries were taken into account and analysed and elaborated in detail. Their results and analyses show the possible impact of the COVID-19 pandemic on the performance, productivity and relative efficiency of university libraries in Croatia. However, this assumption needs to be further analysed through the application of regression analyses and OLS (Ordinary Least Squares), which is a plan for future work. In addition, the authors plan to include and analyse a longer time period in future work and also include the basic DEA models, as there could be differences in the efficiency results obtained with the window DEA and the basic DEA models. These possible differences could be due to the different characteristics of the models. The WDEA model represents a more advanced and dynamic version of the DEA method which allows for the inclusion of time series data and accounts for dynamic changes in efficiency over time (Xiang-jun & Guang, 2009). Therefore, further research is needed to fully understand the extent of these differences and their impact on decision making and performance analysis of university libraries.

The scientific contribution of this paper is threefold. Firstly, it is the first empirical study dealing with the efficiency and performance of libraries in Croatia (and Europe for that matter). Secondly, there are not many published studies for the period after 2018, so the impact of the COVID-19 pandemic has not yet been addressed. And thirdly, the technique of WDEA has been neglected by researchers and rarely used in the assessment of library performance. Furthermore, the practical implications of this study are reflected in the new insights it offers to university management, the interested public, but also any other public sector management.

However, this study is not without limitations. First and foremost, one university library was intentionally not included in the sample and analysis (namely the library of the University of Slavonski Brod). The university in Slavonski Brod was founded in 2020, so the statistical indicators for the university library are only known since 2021. Due to the unavailability of data, this library was not included. Secondly, the sample is small and the DEA method is not applicable to such a small sample with three inputs and three outputs. However, this limitation was overcome by using the window technique. The WDEA technique is also criticised by its founders, due to the inability to test and evaluate the first and the last year in the analysis "as frequently as the other analysed years" and this may be its major drawback (Cooper et al., 2007). Thirdly, the time frame of five years an- alysed is relatively short and may not provide a successful overview of the performance of this public sector over a longer period of time. This will be considered in further work covering a longer time frame. Finally, the analysis only covers university libraries in Croatia. In the future, the authors plan to analyse the relative efficiency and performance of libraries in the EU and South East Europe.

This is an original study, since, to the best of the authors' knowledge, there has been no comparable study analysing the efficiency and performance of academic libraries in Croatia. Furthermore, there are only a few studies that deal with the efficiency and performance of university libraries in Europe. Thus, this study represents a springboard for future studies and research endeavours in this and other public sectors.

References

REFERENCES

1. AlKhars, M.A, Alnasser, A.H., AlFaraj, T. (2022). A Survey of DEA Window Analysis Applications". Processes, 10, 1836. https:// doi.org/10.3390/ prioog1836

2. Bawden, D., & Robinson, L. (2015). Introduction to information science. Facet Publishing.chen

3. Chen, T. (1997) 'A measurement of the resource utilization efficiency of university libraries', International journal of production economics, 53(1), pp. 71-80.

4. Fotova Cikovié, K., Lozié, J., & Guzovski, M. (2022). Do mergers and acquisitions improve bank efficiency? Evidence from North Macedonia. Journal of International Studies (2071-8330), 15(2). doi:10.14254/2071-8330.2022/15-2/6

5. Clark, M. (2015) 'Improving library efficiency to meet patron's needs: A data envelopment analysis benchmarking model', in. 2015 Portland International Conference on Management of Engineering and Technology (PICMET), IEEE, pp. 1415-1423.

6. Cooper, W. W., Seiford, L. M., Tone, К. (2007). Data envelopment analysis: A comprehensive text with models, applications, references and DEA-solver software. New York, NY: Springer Science + Business Media, LLC. doi: 10.1007/978-0387-45283-8

7. Cooper, W., Seiford, L. & Tone, K. (2007). Data Envelopment Analysis, A Comprehensive Text with Models, Applications, References and DEA-Solver Software, 2nd ed.; Springer Science-i-Business Media, LLC, New York, NY, USA.

8. Cvetkoska, V., & Cikovi, K. F. (2020). Assessing the relative efficiency of commercial banks in the Republic of North Macedonia: DEA window analysis. Croatian operational research review, 217227. DOI: 10.17535/crorr.2020.0017

9. De Carvalho, F. A., José Jorge, M., Filgueiras Jorge, M., Russo, M., & Oliveira de Sá, N. (2012). Library performance management in Rio de Janeiro, Brazil: Applying DEA to a sample of university libraries in 2006-2007. Library Management, 33(4/5), 297-306.

10. De Groote, S., & Scoulas, J. М. (2021). Impact of COVID-19 on the use of the academic library. Reference Services Review, 49(3/4), 281-301.

11. EBLIDA. (2020) A European Library Agenda for the post-Covid 19 age [work in progress] Draft May, 2020", available at: http://www. eblida.org/ Documents/EBLIDA-Preparing-a-European-library-agenda-for-the-post-Covid-19-age.pdf. (Accessed: 4. October, 2023)

12. El Refae, С. A, Kaba, A., & Eletter, S. (2021). Distance learning during COVID-19 pandemic: satisfaction, opportunities and challenges as perceived by faculty members and students. Interactive Technology and Smart Education, 18(3), 298-318.

13. Kao, C. and Lin, Y. (2004) 'Evaluation of the university libraries in Taiwan: total measure versus ratio measure', Journal of the Operational Research Society, 55(12), pp. 1256-1265.

14. Kao, C. and Lin, Y.-C. (1999) 'Comparing university libraries of different university size', Libri,49, 150-158

15. Klopp, G., (1985). The analysis of the efficiency of production system with multiple inputs and outputs. Ph.D. Thesis, (Chicago, University of Illinois at Chicago, Industrial and Systems Engineering College).

16. Lauri Korajlija, A. Jokic-Begic, N. (2020). COV1D-19: Concerns and behaviours in Croatia. British journal of health psychology, 25(4), 849-855.

17. Lee, CW, Kwak, N., Garrett Jr, W.A. (2013) 'A comparative appraisal of operational efficiency in us research-university libraries: A dea approach', in Applications of Management Science. Emerald Group Publishing Limited, pp. 117-130.

18. Lynch, B. P. (2017). Academic librarianship today. Rowman & Littlefield.

19. Martzoukou, K. (2021.) 'Academic libraries in COVID-19: a renewed mission for digital literacy'. Library management[online], 42(4/5): the worst case scenario after COVID, pages 266-276.

20. Mohamed Shahwan, T, Kaba, А (2013). 'Efficiency analysis of GCC academic libraries: an application of data envelopment analysis'. Performance Measurement and Metrics 14 (3), 197-210. DOI 10.1108/PMM-07-2013-0023

21. Nacionalna i sveuéilisna knjiznica u Zagrebu. Arhiva statistiékih podataka i pokazatelja uspjeSnostifonline], Available at: http://maticna.nsk. hr/wpcontent/uploads/2023/05/Podaci_sveuc_ vsk_znan_2022_10052023 xlsx. (Accessed: September 2023)

22. Najaf, A, Emamgholipour Sefiddashti, S., Sheikhshoaei, F., Razavi Hajiagha, S. H., & Masoumi, D. (2020). DEA-based Performance Evaluation of Libraries: A Systematic Mapping study". Investigación bibliotecológica, 34(85), 227-251.

23. Noh, Y. (гот). 'Evaluation of the resource utilization efficiency of university libraries using DEA techniques and a proposal of alternative evaluation variables'. Library Hi Tech, 29(4), pp. 697- 724.

24. Paradi, J.C, Sherman, О. H., Tam, F.K. (2018). Data Envelopment Analysis in the Financial Services Industry. A guide for practitioners and ana- lysts working in Operations Research using DEA, 266. Springer. DOI: https://doi.org/10.1007/9783-319-69725-3

25. Peykani, P., Farzipoor Saen, R., Seyed Esmaeili, F. S., & Gheidar-Kheljani, J. (2021). 'Window data envelopment analysis approach: A review and bibliometric analysis'. Expert Systems, 38(7), e12721. https://doi.org/10.1111/exsy.12721

26. Pjevèevié, D., Radonjié, A, Hrle, Z., Colié, V. (2012). 'DEA Window Analysis for Measuring Port Efficiencies in Serbia'. Promet - Traffic&Transportation, 24 (1), 63-72

27. Reichmann, G., Sommersguter-Reichmann, М. (2006). Valverde-Berrocoso, )., Fernández-Sánchez, M. R., Revuelta Dominguez, F. |, & Sosa-Díaz, M. J. (2021). The educational integration of digital technologies preCovid-19: Lessons for teacher education. PloS one, 16(8), e0256283. "University library benchmarking: An international comparison using DEA', International journal of production economics, 100(1), pp. 131-147.

28. Repkova, I. (2014). Efficiency of the Czech banking sector employing the DEA window analysis approach. Procedia Economics and Finance, 12, 587-596. DOI: https://doi.org/10.1016/S22125671(14)00383-9

29. Ministry of Culture and Media (2020). Smjernice га rad 5 gradom i korisnicima Nacionalne i sveuciliène knjiZnice u Za

30. grebu od 27. travnja 2020. [Online]. Available at: https://min-kulture.gov.hr/vijesti-8/normalizacija-rada-u-arhivima-knjiznicama-muzejima-i-galerijama/ smjernice-za-rad-s-gradjom-i-korisnicima-nacionalne-i-sveucilisne-knjiznice-u-zagrebu-od27-travnja-2020/19424. (Accessed: September 2023)

31. Standard za visokoSkolske knjiZnice (2022). [Online]. Available at: https://narodne-novine.nn.hr/ clanci/sluzbeni/2022_07_81_1182.html. (Accessed: September 2023)

32. Tavares, В. S., Drumond, С. M., Angulo Meza, L., & Mexas, M. P. (2018). 'Efficiency assessment in university libraries'. Transinformacáo, 30, 65-79.

33. Valverde-Berrocoso, J., Fernández-Sánchez, M. R., Revuelta Dominguez, F. I., & Sosa-Díaz, M. J. (2021). The educational integration of digital technologies preCovid-19: Lessons for teacher education. PloS one, 16(8), e0256283.

34. Wakeling, S., Garner, J., Hider, P., Jamali, H., Lymn, J., Mansourian, Y., € Randell-Moon, H. (2022). The challenge now is for us to remain relevant: Australian public libraries and the COVID-19 crisis". IFLA journal, 48(1), 138-154.

35. Xiang-jun, L. |, & Guang, W. А. N. С. (2009). A Review Study on Two Basic Models of DEA. Journal of Xian University of Arts & Science, 12(2), 67-71.

36. Zakon o knjiznicama (2019). [Online]. Available at: https: https://narodne-novine.nn.hr/clanci/sluzbeni/2019_02_17_ 356.html (Accessed: September 2023)