The Medical Library Association Benchmarking Network: results*

Abstract

This article presents some limited results from the Medical Library Association (MLA) Benchmarking Network survey conducted in 2002. Other uses of the data are also presented. After several years of development and testing, a Web-based survey opened for data input in December 2001. Three hundred eighty-five MLA members entered data on the size of their institutions and the activities of their libraries. The data from 344 hospital libraries were edited and selected for reporting in aggregate tables and on an interactive site in the Members-Only area of MLANET. The data represent a 16% to 23% return rate and have a 95% confidence level. Specific questions can be answered using the reports. The data can be used to review internal processes, perform outcomes benchmarking, retest a hypothesis, refute a previous survey findings, or develop library standards. The data can be used to compare to current surveys or look for trends by comparing the data to past surveys. The impact of this project on MLA will reach into areas of research and advocacy. The data will be useful in the everyday working of small health sciences libraries as well as provide concrete data on the current practices of health sciences libraries.

Headnote

Objective: This article presents some limited results from the Medical Library Association (MLA) Benchmarking Network survey conducted in 2002. Other uses of the data are also presented.

Methods: After several years of development and testing, a Web-based survey opened for data input in December 2001. Three hundred eighty-five MLA members entered data on the size of their institutions and the activities of their libraries. The data from 344 hospital libraries were edited and selected for reporting in aggregate tables and on an interactive site in the Members-Only area of MLANET. The data represent a 16% to 23% return rate and have a 95% confidence level.

Results: Specific questions can be answered using the reports. The data can be used to review internal processes, perform outcomes benchmarking, retest a hypothesis, refute a previous survey findings, or develop library standards. The data can be used to compare to current surveys or look for trends by comparing the data to past surveys.

Conclusions: The impact of this project on MLA will reach into areas of research and advocacy. The data will be useful in the everyday working of small health sciences libraries as well as provide concrete data on the current practices of health sciences libraries.

INTRODUCTION

The need to report the activities of nonacademic health sciences libraries by gathering statistics has been discussed since the early 1980s. Various regional efforts have taken place [1], but the actual measures of activity have not been reported for a national survey since 1972 [2]. The development and implementation of the Medical Library Association (MLA) Benchmarking Network is reviewed in a companion article [3]. This development has now produced the first set of statistical measures of library activity in one class of nonacademic libraries, the hospital library. Efforts of the initiative have expanded to other types of libraries. This paper reports the results of the Benchmarking Network 2002 survey and demonstrates various uses of the data.

BACKGROUND

In an economic climate of managed care and cost cutting in health care, hospital libraries have come under pressure to cut their programs. Some libraries have been eliminated altogether. In 1999, the MLA Board formed the Benchmarking Task Force to develop a way to assist libraries nationwide in gathering comparative statistics. This effort involved many teams and many volunteer hours on the part of MLA members and specific cost outlays in terms of staff and contracts by the association, as reviewed in Dudden [3].

METHODOLOGY

In the summer and fall of 2001 and during the data entry period, all MLA members were asked to submit their data unless their data were included in the Association of Academic Health Sciences Libraries (AAHSL) survey [4, 5]. The Web-based data entry form was open for data collection between December 15, 2001, and March 4, 2002, in the Members-Only section of the MLANET Website. The questionnaire was developed starting in 1999 and beta-tested by seventythree members during a four-month period in early 2001.

A total of 385 MLA members submitted data via the Web. Participants were from each of the fourteen MLA chapters. Thirteen participants were eliminated because they provided no data in the measures section. Two were eliminated because they were AAHSL libraries. Twenty-six more were excluded because they were not hospital libraries. Eight libraries were from research institutions and 18 from other types of special health sciences libraries. Because these libraries had no bed size or other parameters of size comparable to hospital libraries, the team decided to restrict the analysis to hospital libraries, leaving 344 participants. While there is no definitive source for the number of hospital libraries in the United States and Canada, 3 estimates were located. Wakeley reported 2,167 hospital libraries in 1990, which would mean a 16% return [6]. In 2003, the MLA Hospital Libraries section (HLS) had 1,388 members, which would present a 25% return [7]. According to data requested from the National Network of Libraries of Medicine in April 2004, 1,929 hospital libraries were full members of the network (DOCLINE participation required) and 982 hospital libraries were affiliate members (no requirement for size or staffing), for a total of 2,911. If the total number were used, the return rate would be 12% for all libraries and 18% for full members [8].

The task force reviewed the submitted data for accuracy. If it was determined that the participant might have misunderstood the question, the librarian was called. If the librarian had not completed the questions about the hospital such as bed size or number of admissions, these numbers were obtained from the latest edition of the American Hospital Association (AHA) Guide to the Health Care Field [9]. Participants were not required to answer every question. After the fact, the task force decided that every record had to have hospital bed size and library full-time equivalents (FTEs). The task force members called those librarians who did not report these numbers. The questions for the Benchmarking Network 2002 survey and the accompanying definitions are provided in Appendixes A and B, available only online <http://www.pubmedcentral .nih.gov/tocrender.fcgi?action=archive&journal=93>. Data on seventy-three measures of activity were collected as well as twelve parameters of size.

Figure 1Creating a symmetrical distribution for comparing values for a set of classes; hospital full-time equivalents (FTEs) were used for this demonstration: full distributionFigure 2Creating a symmetrical distribution for comparing values for a set of classes; hospital FTEs were used for this demonstration: with 5% eliminated

No outliers, large or small numbers, were eliminated from the parameters. The measures, however, were edited by eliminating outliers at a natural break. Within the 73 measures, each with a possible number of answers of 344, fewer than 50 numbers were removed. The edited data were finalized and sent to the Outcomes Team by June 2002 for analysis and display of the results as described below in this article.

Development of aggregate tables

Throughout the project, the task force has been asked: "Why do you want to have this data? What is your question?" The response to "why" is to have data available if asked by an administration to prove that the library operations are similar to other libraries of comparable size or to improve services through benchmarking and process improvement. The response to "What is your question?" is that everyone has a different question and the data need to be prepared to answer as many questions as possible.

One such question is: What do librarians do? For instance, "On average, how many monographs does a hospital library circulate?" Of the 242 libraries that answered that question, on average, 1,596 monographs were circulated. Then the problem arises that very large and very small libraries distort the average. One librarian would say, "I have 4 FTEs in my library and I circulate 2,835," while another might say, "I work in a one-person library and I circulate 471." The average of all libraries is not that useful when the size of the library varies so much. Tables with parameters of size combined with measures of activity to display the data on the MLANET Members Only Website were developed to take this variation into account. The quartile tables presented in the Survey of Academic and Special Libraries were used as a model but expanded to eight rows with the quartile in the middle [10].

Twelve parameters of size were used for the hospital, training programs, and the hospital library itself. A statistical software program made distributions of the parameters data. Each distribution represented a specific group of numbers. The team needed to determine if the data were distributed as a bell curve or were, at least, distributed symmetrically. As demonstrated in Figures 1 to 3, with so many extreme outliers, distributing the data symmetrically was not possible. The team did not want to exclude any of the participants because they were outliers. The quartile tables became a system of tables divided into eight rows.

Figure 3Creating a symmetrical distribution for comparing values for a set of classes; hospital FTEs were used for this demonstration: with 25% eliminated

Figure 4Percentage of participation in the Benchmarking Network 2002 survey based on the eight ranges of library total FTEs

First, the top and bottom 2.5% of the data or the outliers were identified, as demonstrated in Figure 2, and a more symmetrical distribution curve was obtained. Figure 3, with 25% of the outliers eliminated, demonstrates a more symmetrical curve. A third distribution was developed on the remaining numbers, and quartiles were established within the distribution. Tables could then be developed with 8 rows. On the top and bottom rows of the table were the 2.5% extreme outliers. On the next top and bottom rows were the 10% outliers and in the middle were the remaining libraries set in quartiles of approximately 60. This allowed 75% of the respondents to be divided into quartiles that are similar, and the outliers were represented, not eliminated.

Other distributions were also used. The AHA industry-standard category was modified slightly and used to distribute bed size. Distributions needed to be logical. Distributed quartiles were not logical when using the number of library FTEs as a parameter, because 53% of the respondents had library FTEs between 1 and 2.49. In the data, 80 libraries had exactly 1 FTE and 9 libraries had 1.1 to 1.3. To be able to better analyze the group called a "one-person library," the team decided to break the tables at 1.4. Figure 4 shows the percentage of participation based on the 8 ranges of library FTE. The figure shows a reasonable distribution, even if not exact, with 25% on the top and bottom and 75% in the middle as described above.

The 73 measures of library activity, as listed in Appendix A, were reviewed and put into 5 groups: administrative services (financial), administrative services (staffing and other), public services, technical services, and special services. The goal of the Web-based report was to have a table that matched each of the 73 measures with each of the 12 parameters or 876 tables. Each table would show the 8-part distribution and the number of respondents or qualified answers, the mean, median, third quartile, maximum, and minimum. The third quartile, or 75% number, was added following comments from librarians who participated in quality improvement programs, which preferred to measure against the 75% number and not the mean.

Excel macros were developed by an outside firm to produce the tables. Each of the 12 parameters was placed in a distribution. The measures were divided into the 5 groups mentioned above. A library volunteer was trained to produce the tables using the macros. Each group of measures was run against each parameter, producing 60 Excel spreadsheets, each with 6 to 10 measures, for the total of 876 individual tables. The 60 spreadsheets were exported as simple hypertext markup language (HTML) tables and sent to MLA headquarters where the research and information systems group gave them a consistent look and feel for the Website. They were available to all MLA members by September 13, 2002.

While these tables are still available on MLANET to members at the time of this writing and most likely will be part of the MLA archives, the authors have observed that few of the hospital library surveys done in the past were readily available for use at the time they were done. Of the twelve surveys highlighted in Van Toll's article, "Hospital Library Surveys for Management and Planning: Past and Future Directions," only five were in easily available publications [1]. The surveys were neither widely known nor easy to find even at the time they were completed. Table 1 represents the median value for most of the survey questions. Due to size restrictions, not all answers are represented, but they are all available on the Web. By publishing these median data in a major publication, the data will be widely available for future researchers. The median number was chosen because wide dispersion between the minimum and maximum can often distort the mean. The hospital total FTE parameter was chosen because there was a significant correlation with measures of library size such as space (0.54), budget (0.75), and staff (0.73) (Table 8). Table 1 reflects the state of hospital libraries in 2001 with data gathered in the Benchmarking Network 2002 survey between December 2001 and March 2002.

Development of the interactive site

An interactive site where individuals could select parameters and measures and obtain a list of matching libraries for benchmarking use was a major goal of the project. The aggregate tables served as a template for an interactive site. A contract was given to an outside firm, Ego-Systems, and an interactive site was developed by February 2003. All the libraries that participated in the survey have access to the interactive site. Other MLA members can purchase access.

Once on the interactive site, users land on the Benchmarking Network Report Selection page. Here begins a three-step process. In step one, a time period and/ or a geographic area is chosen. In step two, parameters of size are chosen. The library's data are displayed so users can see how they answered the question. Users can choose to use system hospitals only, single hospitals only, or neither or select teaching hospital, nonteaching hospital, or neither. They can choose a range from any of the twelve parameters of size. If they choose too many, most likely they will get no matches. If less than five institutions meet the selected criteria, the institutions are not identified due to privacy of the data. To increase the number of matching institutions, users need to go back to the selection page and choose fewer criteria. Step three allows users to choose the area of measures they want to see: administrative services (financial and other areas), public services, technical services, or special services. The results include a list of institutions that can be used for benchmarking. Appendix A of the companion article describes how to use the list of libraries for benchmarking projects [3].

Sample size

The already reported response rate of 16% to 23% represents a good sample of hospital libraries nationwide. How can the quality of the sample be judged? Table 2 compares the Benchmarking Network 2002 survey participants to the number of AHA hospitals in the bed-size categories the AHA uses, as collected in the 2002 edition of Hospital Statistics [11]. The sample size of large hospitals is well represented, and the smallest hospitals did not participate in the Benchmarking Network 2002 survey. In the 100-to-199-bed category, 69 libraries reported and 1,439 hospitals. This is an approximately 5% sample size (69/1439 = 0.0479) of all hospitals, whether or not the hospital has a library. For hospitals over 200 beds, the sample size was 10% to 27% of the AHA hospitals. This is a more than adequate sample size.

Participants

Again using the data from the Benchmarking Network 2002 survey and AHA's Hospital Statistics 2002 [11], the percentage of participants in the survey and the percentage of beds in each AHA bed-size category can be compared, as shown in Figure 5. Using the 8-bed size categories for AHA's 5,810 hospitals, the percentage of hospitals and survey participants in each category was determined. The bed-size category of 100-199 has the strongest match, containing 24.8% of hospitals and 20% of library participants. No libraries reported in the under-50-bed-size hospitals, which made up 23.5% of all hospitals. Hospitals over 300 beds made up 15.8% of all hospitals, whereas in the Benchmarking Network 2002 survey, 53% of participants were in this range.

Various uses of the data

The Benchmarking Network 2002 survey data can be used in many ways. The data can be used to answer a specific question, to start an internal review of a library process, to perform traditional benchmarking, and to identify benchmarking partners. The data could also be used to answer specific questions or to test a hypothesis or do research. Other uses could be refuting or updating previous surveys or comparing the data to other surveys, either in the present or from the past.

Answering a question using the reports. Many times, librarians are asked to provide data by administrators or they just wonder how they compare on a single activity. A sample question might be, "What is the average number of monographs held in various sizes of institutions compared to my institution?" Table 3 and Figure 6 demonstrate the 2 reporting systems of the Benchmarking Network. Table 3 shows an 8-row table from the aggregated tables, and Figure 6 shows a result from the interactive site. Assume a librarian's institution has 565 beds; on the aggregate table, it falls in range 6 (500-749 beds). Forty-nine matching libraries are on the aggregate table and 53 on the interactive site. (The numbers are different between the two reports because two different computer programs were used.) The interactive site also produces a graph. This library holds 3,000 monographs, as seen on the interactive report, and the median number for this group is 3,061 on the aggregate table and 3,600 on the interactive site. The mean is a little higher, 4,221 on both sites, distorted by the outliers. Note that the middle 4 rows of the aggregate table, divided into quartiles, have between 30 and 70 libraries represented. The interactive site also includes a list of the names of the 53 hospitals that matched, which could be used in outcomes benchmarking as described below.

Table 1Aggregate table of various library activity measures (median values) in each of eight ranges of total hospital full-time equivalents (FTEs)

Figure 5Benchmarking Network 2002 survey participants and American Hospital Association (AHA) member hospitals in 2002*: comparison of the percentage of participants and the percentage of beds in each AHA bed size category

Performing internal process reviews. The literature on how to do benchmarking has been reviewed by Todd Smith and Markwell [12]. The authors differentiate between classic or "process" benchmarking and "outcomes" benchmarking. While this kind of classic benchmarking and service review goes on, it is not often reported as benchmarking in the literature. A librarian can now use the Benchmarking Network 2002 survey results to improve internal processes by finding benchmarking partners on a national level as described in Appendix A of the companion article [3]. Written by Todd Smith and Markwell, the MLA Benchmarking Network Survey Participant's Guide to Finding Benchmarking Partners lays out a step-by-step process for finding benchmarking partners and starting a process using the interactive site. The team hopes that any libraries doing this will report their experience in the literature.

Performing outcomes benchmarking. Outcomes benchmarking can be done by libraries participating in the survey as a group and then purchasing and using the data in spreadsheet format to analyze their situation as compared to the whole of the survey or parts of the survey. Two good examples of benchmarking done before the Benchmarking Network 2002 survey demonstrate how the survey would have helped with outcomes benchmarking. Good win describes a successful but difficult process, wherein she had to gather all her own statistics and make her own decisions on which data to gather [13]. Harris reports afterward that, like Goodwin, his project lacked clear guidelines from the administration and his imposed timeline was too short [14].

Table 2Sample size of the Benchmarking Network 2002 survey participants based on American Hospital Association (AHA) bed size categories*

Table 3Comparison of the Benchmarking Network 2002 survey reporting systems: aggregate table: number of print monographs held for each of 8 ranges for number of staffed beds

Using the Benchmarking Network 2002 survey, the Northern and Southern California Kaiser Permanente libraries successfully completed such a project. Twenty-six Kaiser libraries participated in the survey as a coordinated group, as reported by Bertolucci and Van Houten at MLA '02 and MLA '03 [15, 16]. The eleven libraries in the Northern California district analyzed the data and presented their finding to the administration. They compared three items from their budgets-books, journals, and staff-to the median benchmarking data for other libraries of like size. They then submitted a request for additional funding, presenting the discrepancy they found between the funding for those budget items for the Kaiser libraries and the median for libraries documented in the benchmarking data as evidence for it. The analysis showed that funding of book and journal expenditures and staff for other libraries, as reported in the benchmarking data, were significantly higher than for these items in the Kaiser library budgets. When requesting a budget increase, the librarians developed scenarios of how additional money would be spent using three different funding levels. The medium scenario, which increased funding by one million dollars for the eleven libraries, was subsequently approved.

Testing hypotheses. The Benchmarking Network 2002 survey data can be used to retest a hypothesis. A question might be: "How does the number of total print serials compare to the number of interlibrary loans borrowed?" The theory being tested would be that the more serials a library has, the fewer interlibrary loans (ILLs) have to be borrowed, in other words, the number of subscriptions owned is negatively correlated to with the number of items borrowed. In an article by Dudden, this null hypothesis was tested on local data [17]. That study, with 50 libraries in Colorado and Wyoming, showed a marginally positive correlation of 0.29 (P = 0.523). The null hypothesis was contradicted in that study.

Figure 6Comparison of the Benchmarking Network 2002 survey reporting systems: output from the interactive site: number of print monographs: 53 hospitals matched criteria of number of staffed beds of 500 to 749

Table 4Computer resources and services in hospital libraries reported in the Benchmarking Network 2002 survey

As shown in Figure 7, testing the same hypothesis in the Benchmarking Network 2002 survey, with 315 libraries answering the questions, shows a significant positive correlation 0.057 (P < 0.0001). The null hypothesis is again contradicted. These findings were supported by two other studies that showed that purchasing more journal subscriptions did not result in a decrease in ILL borrowing [18, 19]. Based on the results of these three studies and Figure 7 from the Benchmarking Network 2002 survey, it can be suggested that purchasing more journal subscriptions would most likely not significantly decrease ILL requests.

Testing statements from previous surveys. The data can also be used to prove statements in previous studies to either be inaccurate or have changed over time. While talking about "highly advanced, highly wired" libraries, a 1999 study refers to "a vast underclass of hospital libraries that have very much fallen behind the times." The survey of academic and special libraries quoted here surveyed 130 libraries, of which 22 were hospital and other health care libraries [10]. Looking at the Benchmarking Network 2002 survey data to see how "highly wired" the 344 hospital libraries are in 2001, some of the special services questions that related to computer and Internet use in the library were analyzed. Table 4 shows that 99% of the respondents have computer workstations in the library, 57% have online public access catalogs, 71% have library Web pages, and 33% support Web design. On another aggregate table, 264 libraries reported purchasing an average of 232 electronic full-text journals for their users. As stated above, the sample size for hospitals over 200 beds was between 10% to 27% of the AHA hospitals. Based on this larger survey, the authors would challenge the finding of the previous survey and comment that these hospital libraries were remarkably "wired." They either changed remarkably in two years or the previous authors came to their conclusion with too small a sample.

Comparing with other current surveys. In the future, the Benchmarking Network Editorial Board (BNEB) will attempt to merge the MLA data with comparable AAHSL data. In 2002, the authors were given access to the AAHSL data and found some comparable aggregate numbers. Tables 5 through 7 demonstrate some of these comparisons. It is interesting that the ILL lending activity for the 344 small libraries represented a little more than half as much as the 131 large academic health sciences libraries represented in a AAHSL survey. Other numbers reflected the different collection emphasis of the different types of libraries. As a group, the health sciences libraries represented in the 2 surveys hire personnel and purchase products with a combined expenditure of over $408 million.

Another type of question is: "How does the percentage of support staff compare between the two types of libraries?" While the academic libraries do have a larger percentage, it is not as different from hospital libraries as might be assumed. "How does the percentage of total expenditures spent on print serials compare?" Again the percentages are remarkably close when it would seem hospital libraries would not be able to spend as much.

Comparing to past studies. A major hospital library survey from the past was the 1980 Kentucky Ohio Michigan Regional Medical Library (KOMRML) hospital library survey [20]. In a survey of 596 hospitals in Kentucky, Ohio, and Michigan identified in the AHA Guide to the Health Care Field, 360 questionnaires were returned for a 60% return rate. Of these, 311 libraries reported data, but 49 had no libraries. The survey committee reported 4 correlations using the data they collected. Table 8 compares the KOMRML correlations with correlations from the Benchmarking Network 2002 survey.

Table 5Comparing the Benchmarking Network 2002 survey with the AAHSL 2002 survey: What is the combined total number of service requests and resources in selected categories?

Table 6Comparing the Benchmarking Network 2002 survey with the AAHSL 2002 survey: How does the percentage of support staff compare between the two types of libraries?

In 1980, the number of beds was the standard count for the size of a hospital. So the correlations in the KOMRML survey for bed size were very strong, with more than 0.5 in all cases and 0.76 for library staff. In the Benchmarking Survey, this was no longer true, with 0.41 to 0.43 being the correlations. In 2002, library activity correlated more to hospital FTEs. This is a statistical example of what most people already know. Hospitals now have large outpatient activities and other enterprises that make total FTE a more reasonable number to justify library size than the number of beds. Other interesting changes have taken place, such as the change in the correlation between budget and square footage (0.36 changed to 0.70) and budget and library FTEs (0.84 changed to 0.94). Does this mean libraries have more space? Does this mean salaries are a larger part of the budget than 20 years ago?

Using the survey to develop standards. These same kinds of correlations were done to assist the MLA Hospital Libraries section Standards Committee in developing a formula for library staffing in their 2002 standards [21, 22]. The Outcomes Team worked with the HSL Standards Committee in the spring of 2002 to develop these numbers [16]. The formula is "Total institutional full time equivalents (FTE)/700 = minimum library FTE," where the FTE includes the medical staff. Qualifications fare also provided for extra services provided by the library. In Table 8, total hospital FTE has a more significant correlation than number of beds, physicians, or residents. The index factor of 700 was also developed using the data from the Benchmarking Survey, comparing the total hospital FTE indexed by 700 with the reported staffing in the libraries.

DISCUSSION

As demonstrated in the various analyses above, having data on the activities of nonacademic health sciences libraries provides many avenues for research and advocacy. While it is not expected that hospitals under fifty beds will support a library, increasing the number of hospital library participants in the fifty-to-ninetynine-bed range will need to be addressed in future surveys. Small research projects internal to the library operation can be accomplished using this comparative data. An educational effort needs to be made so that members can learn to use the data more efficiently for these small research projects or benchmarking. MLA has been offering continuing education courses on benchmarking for the last few years. The team hopes that librarians will report their research and projects in the literature. As outlined in MLA's research policy, Using Scientific Evidence to Improve Information Practice, even these small research projects should be a part of a librarian's self-improvement [23].

If a librarian has a simple question and wants to know what others do in an area, it is possible that the aggregate tables or the interactive site can supply an answer. Outcomes benchmarking can be planned in advance, and the Benchmarking Network can be used to gather data. Many assumptions are made about small health sciences libraries, and these assumptions can now be tested with this data. Surveys from the past can be compared to look for trends. In the future, these trends can be studied with the benchmarking data as more surveys are done. While using numeric guidelines in standards is controversial, developers of standards can certainly use the survey results as a guide to best practices.

CONCLUSION

The various uses of the data presented in this article demonstrate the importance of the MLA Benchmarking Network data to research efforts in medical librarianship. MLA has done survey research into the salaries of medical librarians since 1983 with a program of triennial surveys, the most recent reported by Wallace [24]. AAHSL has done surveys of library activity since 1975. The AAHSL data serve the members "as a highly regarded and essential management tool." [5] The annual surveys have become part of the culture of AAHSL. The success of this program serves as an example for MLA. The triennial salary survey has become part of MLA's culture. The Benchmarking Network surveys could also become part of MLA's culture. While the diversity of MLA's nonacademic members poses a continuing challenge, a supported program of MLA member surveys has a great potential for research and advocacy and as an individual library management tool.

Table 7Comparing the Benchmarking Network 2002 survey with the AAHSL 2002 survey: How does the percentage of expenditures on print serials compare between the two types of libraries?

Figure 7Correlation between the number of items borrowed and the number of current print journal subscriptions using the MLA Benchmarking Network 2002 survey: borrowed items to current journal subscriptions, significantly positive correlation 0.57 (P < 0.0001)

Table 8Correlation scores among selected survey questions on the Benchmarking Network 2002 survey and the Kentucky Ohio Michigan Regional Medical Library (KOMRML) 1980 survey*

Footnote

* Based on presentations at MLA '02, the 102nd Annual Meeting of the Medical Library Association; Dallas, TX; 2002; and MLA '03, the 103rd Annual Meeting of the Medical Library Association; Orlando, FL; 2003.

References

REFERENCES

1. VAN TOLL FA, CALHOUN JG. Hospital library surveys for management and planning: past and future directions. Bull Med Libr Assoc 1985 Jan;73(1):39-42.

2. MILLER JD. Health sciences libraries in hospitals. Bull Med Libr Assoc 1972 Apr;60(2 suppl):19-28.

3. DUDDEN RF, CORCORAN K, KAPLAN J, MAGOUIRK J, RAND D, TODD SMITH B. The Medical Library Association Benchmarking Network: development and implementation. J Med Libr Assoc 2006 Apr;94(2):107-17.

4. BYRD GD, SHEDLOCK J. The Association of Academic Health Sciences Libraries annual statistics: an exploratory twenty-five-year trend analysis. J Med Libr Assoc 2003 Apr; 91(2):186-202.

5. SHEDLOCK J, BYRD GD. The Association of Academic Health Sciences Libraries annual statistics: a thematic history. J Med Libr Assoc 2003 Apr;91(2):178-85.

6. WAKELEY PJ, FOSTER EC. A survey of health sciences libraries in hospitals: implications for the 1990s. Bull Med Libr Assoc 1993 Apr;81(2):123-8.

7. MEMBERSHIP COMMLTTEE, HOSPITAL LIBRARIES secTION, MEDICAL LIBRARY ASSOCIATION. HLS Membership Committee annual report 2002-2003. [Web document]. 2003. [cited 1 Mar 2005]. <http://hls.mlanet.org/Organization/reports03/ membership03.pdf>.

8. HAMASU C. Personal communication. Salt Lake City, UT: National Network of Libraries of Medicine. Midcontinental Region, Apr 2004.

9. AMERICAN HOSPITAL ASSOCIATION. American Hospital Association guide to the health care field. Chicago, IL: The Association, 2001.

10. PRIMARY RESEARCH GROUP. The survey of academic and special libraries: a new report from Primary Research Group. 2001 ed. New York, NY: Primary Research Group, 2000.

11. AMERICAN HOSPITAL ASSOCIATION. Hospital statistics. 2002 ed. Chicago, IL: The Association, 2002.

12. TODD-SMITH B, MARKWELL LG. The value of hospital library benchmarking: an overview and annotated references, print and Web resources. Med Ref Serv Q 2002 Fall;21(3):8595.

13. GOODWIN C. Report on a benchmarking project. Natl Netw 1999 Oct;24(2):16-7.

14. HARRIS L. Report from down under: results of a benchmarking project. Natl Netw 2000 Apr;24(4):8-9.

15. FULDA PO, SATTERTHWAITE RK, EDS. Proceedings, 102d Annual Meeting, Medical Library Association, Inc., Dallas, Texas; May 17-23,2002. J Med Libr Assoc 2003 jan;91(1):103-36.

16. FULDA PO, SATTERTHWAITE RK, EDS. Proceedings, 103rd Annual Meeting, Medical Library Association, Inc., San Diego, California; May 2-7, 2003. J Med Libr Assoc 2004 Jan; 92(l):117-54.

17. DUDDEN RF, COLDREN S, CONDON JE, KATSH S, REITER CM, ROTH PL. Interlibrary loan in primary access libraries: challenging the traditional view. Bull Med Libr Assoc 2000 Oct;88(4):303-13.

18. COSTELLO J, DUFFY C. Academic interlibrary loan in New York state: a statistical analysis. J Interlibrary Loan Inf Supply 1990;1(2):41-3.

19. MILLER J. Primary clientele as a predictor of interlibrary borrowing: a study of academic health sciences libraries. Bull Med Libr Assoc 1997 Jan;85(1):11-5.

20. Development of a hospital library survey: a KOMRML Committee approach. Detroit, MI: Kentucky Ohio Michigan Regional Medical Library, 1982.

21. GLUCK JC, HASSIG RA, BALOGH L, BANDY M, DOYLE JD, KRONENFELD MR, LINDNER KL, MURRAY K, PETERSEN J, RAND DC. Standards for hospital libraries 2002. J Med Libr Assoc 2002 Oct;90(4):465-72.

22. HASSIG RA, BALOGH L, BANDY M, DOYLE JD, GLUCK JC, LINDNER KL, REICH KL, VARNER D. Standards for hospital libraries 2002 with 2004 revisions. J Med Libr Assoc 2005 Apr;93(2):282-3.

23. MEDICAL LIBRARY ASSOCIATION. Using scientific evidence to improve information practice: the research policy statement of the Medical Library Association. [Web document]. Chicago, IL: The Association, 1995. [rev. 15 Dec 2000; cited 15 Jan 2004]. <http://www.mlanet.org/research/ sciencel.html>.

24. WALLACE M, MCMULLEN TD, CORCORAN K. Findings from the most recent Medical Library Association salary survey. J Med Libr Assoc 2004 Oct;92(4):465-72.

Received April 2005; accepted November 2005