The Culture of Collection Evaluation in Pakistan
Professor Kanwal Ameen, PhD
"A library is a place where books are kept" is still the ruling perception in both developed and developing worlds. For many people, library collections and libraries are more or less the same. The history of libraries demonstrates that only recently have LIS professionals have turned their attention toward clients. Now they are asked directly about their information needs instead of librarians making assumptions and collecting titles to meet them. This development took place in part because of the need to evaluate collections to get grants as well as the application of marketing ideas in libraries since the late 1970s.
Experts such as Mosher (1979), Lancaster (1988; 1993), Gorman (2000), Clayton & Gorman (2001), and Crawford (2006) have produced valuable literature on collection evaluation. The emphasis on evaluating has grown, to keep balance between inputs and outputs, and to achieve efficiency and effectiveness in managing collections. Recently, the aim of building a huge physical collection has also lost its inherent worthiness.
Nonetheless, one should not view the development of evaluation culture as a global trend. It is apparent from the comparison of the developed and the developing countries' literature that there is a major divide between their practices in this regard: it may be called an “evaluation divide.” A review of the local literature and practices testifies that our libraries have had been living in the collecting era, with a focus on developing, adding to, and increasing the size of collections (Ameen, 2005a). The issues of identifying strengths and weaknesses, effectiveness and use, outputs and outcomes of a collection have not been discussed in the local literature (Ameen and Haider, 2007). The reason behind the absence of evaluation culture might have been the traditional perception about libraries— inherently valuable agencies for society.
With the impact of globalization the situation is destined to change, sooner or later, in Pakistan also. Certain indicators show this trend, such as the concern of university administration, government funding agencies, and clients regarding delivery of desired quality services (e.g., Higher Education Commission (HEC); Tahira, 2008). Since 2006, HEC has started ranking the universities using its eight-point set of standards.
Defining Collection Evaluation
The following definitions have been used for the purpose of this study. They are relatively old, but suitable for studying developing countries' practices.
An Overview of University Libraries in Pakistan
There are 120 HEC recognized universities and chartered institutes of higher education at present, including both private and public sector universities. Public sector universities generate their own funds besides getting HEC grants. The income from students' admission fee covers only a small portion of expenses; there is a very nominal admission fee in public sector universities as compare to very expensive private sector universities/institutions. Only wealthy people can afford to send their children to these universities. However, in all kinds of universities, funding authorities, parent organizations, and the service community are becoming sensitive about quality services. The author has observed this phenomenon as a member of selection board interviewing for university librarian, in job advertisements, and discussions with peers.
Since HEC's establishment in 2002 by the Federal Government of Pakistan, it has been playing a significant role in running and monitoring the working of universities; it offers substantial annual grants to run different projects/plans. The very first objective of HEC Finance Division is:
The prime focus of HEC is to promote the quality academic and research activities. Its Quality Assurance Agency states:
HEC Quality Assurance Agency Project has established Quality Enhancement Cells (QECs) at ten universities, and twenty more QECs will be established in the public sector universities for improvement of their academic, teaching, and learning standards. The QEC will serve as a focal point for quality assurance in higher learning. (http://www.hec.gov.pk/QualityAssurance/QA_Agency/qaagency.htm).
Structure of University Libraries
Universities have administratively different library structure, such as (i) central library with independent, subject libraries attached with the academic units (ii) central library with small reference libraries in academic units (iii) only central library. It may be categorized as centralized and decentralized; (University libraries will be referred as libraries throughout the study).
The study by Ameen (2005) found that in most cases, the authority regarding financial grants, administrative, and policy decisions lies in the hierarchy of the syndicate, academic council, and library committee. A library committee usually consists of chief librarian (secretary/ex-officio member) and nominees of syndicate, academic council, faculty, and administration. The internal management of the library is the responsibility of the chief librarian. Not all universities have not created a library committee. Libraries get annual grants from their parent institutions. The size of grant and allocation formulas varies from university to university .
University library collections are mainly books, followed by serial publications like newspapers, magazines, and journals (Ameen, 2005). They spend a large portion of their funds on books. A sound selection policy and practice play a fundamental role in developing a strong, balanced book collection. Another study by Ameen (2007) reports that a number of factors affect selection practice, such as too much dependence on faculty for selection, constraints of faculty members as selectors, non-availability of selection aids, and lack of awareness regarding online resources available for selection.
Since 2004, HEC National Digital Library has been providing access to 20,000 full text journals from the world's leading publishers and 10,000 e-books. It is a unique program which has a vital role in quality research production by university professors. The literature, peers opinion, the author's 24 years of experience, and observation both as university librarian and academician testifies that these libraries are better equipped than other types of libraries in the country.
The international literature presents many discussions and research on impact, evaluation, assessment, outcome, and accountability of library services. The term evaluation is more popular in the American literature and assessment in the British. It is apparent that all kinds of services are subject to evaluation; however, due to the enormous amount of money spent on collections, they are the most important target for evaluation. Nisonger (1992; 2003), produced two guides to the literature on collection evaluation, illustrating the depth of the literature until 2002. He describes the dichotomies of collection evaluation as “collection-centered vs. client centered, qualitative vs. quantitative, inputs vs. outputs, outputs vs. outcomes, micro-evaluation to macroevaluation” (pp. xvii-xviii).
The literature demonstrates numerous methods for evaluation of collections to determine their effectiveness and output. Evans and Saponaro (2005) give a checklist of internal and external reasons for collection evaluation (pp. 316-17). Emanuel (2002) observes that it is time-consuming but results presented to the subject selector may be used to adjust the vendor profile of a collection, or re-establish contact with the departments being served, and an evaluation can ultimately save the library money.
The current literature stresses user-centered evaluation methods. The study by Agee (2005) establishes that collection evaluation has implications for successful future collection management, financial resource management, and effective format selection. Hiller, et al., (2006) argue that assessment is different in different organizational cultures, and for moving library assessment from a project-based approach to a more programmatic, integrated, and sustainable operation. The Association of Research Libraries in is developing a stronger understanding of the value of assessment and library leadership supports this movement. Markless and Streatfield (2006) explore why we need evidence of the impact of a library through evaluation and discuss different evaluation models (pp. 1-20). With the emergence of a digital paradigm, assessment has widened, covering use and usability of digital collections, websites, and access (e.g. Bollen, 2002; Covey, 2002; Williams et al., 2006).
A l iterature search on developing countries and collection-related issues reveals only handful of works. It reveals that they are far behind in following current practices. Most rely on American and British literature for theoretical understanding and local consumption regarding collection management issues. The literature on collection evaluation from developing countries mostly addresses the issues of book production, acquisition, and insufficient human and financial resources, which are all inputs.
Ahmad (1984) analyzes contemporary procedures, practices, applications, and operations of libraries of eight Arab countries, with a mention of Pakistan as well, focusing specifically on acquisition and selection methods for local and foreign materials. Siddiqui (1995) conducted a study on developing of a model interlibrary loan network among academic libraries of Saudi Arabia. In India, works by Deshpande, (1985), Panda, (1992), and Rout, (1992) deal with issues such as their history, management, finance, and collections. Sood and Sharma's (1991) compilation contains essays on different aspects of university librarianship. Sharma (1986) discusses academic library developments in India before and after independence with suggestions to improve collections. Prasher (2002) produced a lengthy text on India's libraries. The section on collections consists of chapters on policies, procedures, and problems. The state of university librarianship in India is not very different from that of Pakistan and the universities are facing the same kind of problems in collection management.
Local Literature on University Libraries or Collection Issues
The literature on librarianship in Pakistan does not report any practice of formal collection evaluation. An Index of Periodical Literature from 1947 to 1997 compiled by Samdani and Mahmood (1999) has only one work on collection evaluation . A comparatively recent online index compiled by Samdani and Mahmood (2003) has none.
Significant research was done by Moid (1955) on the general state of librarianship. Findings include the fact that the emergence and growth of university libraries in Pakistan has been dependent on the function of each university. The universities concentrate on three categories: examining, teaching, and research (p. 28). Razzaque (1971) states that the local market cannot supply 25 percent of the needs of the libraries and a number of problems are encountered when acquiring foreign books and journals.
Ahmad (1984a) produced a descriptive and critical review of the status of libraries in Pakistan. He discusses things like administrative pattern, selection, acquisition, circulation, loss of material, buildings, technical services, and user instruction. The author concludes that “the vast growth of published literature and an increased pressure caused by overall adverse economic conditions in Pakistan have in their different ways placed libraries and librarians in a very difficult position” (p. 51).
In a comparatively recent work, Khurshid (2000) describes overall developments in library services in Pakistan. With reference to university libraries, he states that when L.C. Key, an expert from Australia, visited Pakistan in 1955-56, he found university libraries to be the most advanced group of libraries in the country. Nevertheless, Key recommended the development of all libraries and justified the need for the open access in libraries. Another report by David G. Donovan in1974 recommended a minimum 5 percent of the budget of a university for its library development, establishment of a Standing Subject Committee on Problems of University Library & Library Education (as cited by Khurshid, 2000). Khurshid discusses book production in Pakistan and problems of library purchases. A better relationship between the book industry and libraries is crucial to run both smoothly. Library automation has emerged as an area to be addressed since 1980s.
Research has been conducted at tje Master's and Ph.D. levels on collection-related issues in libraries of Pakistan at local universities and by Pakistanis having studied abroad. The LIS Department at University of the Punjab (PU) conducted surveys on information needs of faculty from different disciplines. The study by Bashir (1991) was first of its kind in Pakistan to measure the rate of accessibility of the general collection to users at PU. It addresses the issue of collection evaluation in terms of document delivery capability. The techniques used in the study were based on Paul Kantor's model. Results showed “the availability rate of Punjab University Library to be 41.3 percent which is considerably low as compared to other studies conducted in developed countries” (p. 46). Among the recommendations were: regular shelf-reading, stock-taking, improved book selection methods, implementation of circulation rules on faculty, better technical processing, and user orientation programs.
The issue of resource sharing and cooperation are important to librarians in Pakistan. Theoretically, they agree on the benefits of such plans, but practically there have been only nominal attempts at cooperative plans in scientific and special libraries. Khan's (1991) doctoral research on the cooperation among libraries of Pakistan concluded that formal cooperation is almost nonexistent in Pakistan. However, informal cooperation exists in the form of interlibrary loan through reprographic services. Khalid's (1997) Ph.D. study on the library cooperation and networking in Pakistan also reported that in developing countries cooperation exists through personal contacts and on an ad hoc basis. Ameen (2008) explored different barriers in collection sharing among libraries in Pakistan.
Qureshi's doctoral study aimed “to arrive at a set of standards to measure the adequacy of university library resources and services in Pakistan” (1982, p. 142). The collected data was compared against the 1975 ACRL Standards for College Libraries. The comparison showed that American standards were too high for Pakistani libraries. The researcher proposed quantitative standards based on the immediate needs and the resources available in Pakistan. The comparison of the collections against the researcher's proposed minimum standards for collection size showed that only two libraries, PU and Peshawar University met the proposed standards (quantitative) in terms of library collections. No library met the standard for periodicals.
Haider (1993) worked on to find out the constraints on acquiring foreign material for Pakistani libraries. He observes that only a fraction of professional, scientific, and advanced knowledge is produced in Pakistan. Consequently, university libraries depend largely on imported foreign publications to build their collections. It is also true for other types of libraries with some variation. The study suggests certain practical and realistic steps to be taken on the part of governments for the improvement of local market and import procedure to make the flow of acquisitions smooth. Ameen (2008) study found that the situation explored by Haider regarding acquisition issues has not changed much.
In another study, Ameen (2007) establishes that libraries are following conventional selection practices in most cases. Faculty and librarians do not have a balanced role in selection strategy in most of the libraries. Relying on faculty members for making selections, and on local vendors for supplying current books, causes undue delay in acquiring needed titles. There must be better liaison among faculty, library staff, and vendors to make quality selections. Automation must be introduced for the quick identification, selection, and purchase of quality, current titles. Librarians must be delegated more authority and be trusted to play an active role in selection.
An overview of the literature shows that the core issues discussed have been insufficient funds, inadequate size of collections, problems in acquisition, lack of standards, poor planning for automation, importance of resource sharing, shortage of competent professionals, and lack of standards. It seems imperative to throw light on status of collection evaluation and to furnish workable suggestions for establishing the culture of evaluation. Since it is an almost untouched area in literature, the study will create awareness, and recommendations will help create user-centered collection evaluation. The study may serve as a base for further investigation.
Objectives of the Study
The study aims to:
The present study is based on the partial data collected through a questionnaire during late 2003 to 2004 for the doctoral research of the author (Ameen, 2005). The review of literature and consultation with peers were made for developing the self-completed questionnaire. The aim was to collect data from central libraries of 40 major HEC accredited universities from all over Pakistan. They included all types of major universities—public, private, general, subject special (Appendix).
After four attempts in terms of reminders, personal letters, phones calls, and visits, 30 responses were obtained. Later, the researcher visited 16 universities, representing different types, in major cities, and interviewed the chief librarians.
Considering the goal of the study and size of the sample, the data has been reported using simple descriptive statistical analysis. The content-analysis of open-ended questions has been made and categorized following the thematic approach. University libraries were chosen as sample because they are comparatively rich regarding print collections and have good professional staff.
Findings and Discussion
This section presents the data analysis regarding the collection evaluation culture in the responding libraries.
Practice of Formal Collection Evaluation
To determine the status of evaluation culture in libraries, the respondents were first asked about prevailing evaluation practice (collection-centered or/and user-centered). Twenty-nine out of 30 respondents replied to the question. It revealed that only 9 (31 percent) libraries would make some kind of evaluation while 20 (69 percent) would not make any kind of evaluation of the existed collection.
Methods Used for Evaluation
An open-ended question was asked to explore the methods used by the responding libraries. The analysis of nine affirmative responses illustrates that the respondents did not differentiate between the formal and informal methods of collection evaluation. The comments by one library were too vague to be included in the analysis and thus excluded. The content-analysis of the remaining answers (n= 8) has been categorized in Table 1.
Table 1: Frequency of Use of Collection Evaluation Methods
The table illustrates that two libraries would practice a kind of user-centered formal method, i.e. seeking their opinion about collection through survey forms, 3 libraries use methods which may be more likely called informal feedback or comments from users. Another library would get overall evaluation of their services using electronic forms. Collection-centered here means that the staff would check for the missing titles, circulation statistics, weeding of titles and replacing needed titles.
Reasons for not Evaluating Collection
In case of the negative response to the first question, they were asked about the reasons for not evaluating collection. Table 2 shows the analysis of reasons given by the other 20 respondents for not evaluating collections.
Table 2: Reasons for not Evaluating the Collection (n = 19)
Only one library stated that it was not necessary to do so while 7 other stated that they never thought about evaluating. Another 9 replied that they did not evaluate collections because of the lack of staff. Two libraries supplied their own statements for not evaluating i.e., careful selection by using standard selection sources to have best collection; because library remains open throughout the year. Though a vast majority of libraries was not involved in making formal evaluation, yet, they did not choose to state that it is not important. It shows that despite agreeing with the importance of collection evaluation, they do not follow the practice. It might be due to lack of staff, know-how, resources, time etc.) An in-depth probe is needed in this regard. Haider (2006) demonstrated that the concept of collection management in its true sense is yet to be developed in today's Pakistan (p. 95). The same is true about whys and hows of collection evaluation.
Seeking User Feedback
Besides making formal evaluation, the role of users in collection management was again asked in getting their feedback. All (N= 30) respondents provided reply to the question. A majority of libraries would not make formal evaluation of collection, but 20 libraries supplied a positive response to the question on getting feedback or suggestions from users about existing collection. The analysis of responses to the open-ended question in this regard demonstrates the use of various methods as follows:
Table 3: Frequency of Methods used for Seeking Feedback about the Collection
Conclusion and Recommendations
The study reveals that mostly libraries seek informal feedback and evaluation. They do formal and/or informal meetings to seek faculty opinion. Only five libraries obtain student opinions in a formal manner. The data clearly demonstrates that making formal, planned evaluation of the existing collection is a seriously overlooked aspect in the process of collection management in university libraries in Pakistan. As mentioned, these are the comparatively developed type of libraries, yet they are hardly following formal evaluation practice. The findings are indicative of the overall absence of culture of collection evaluation in libraries in Pakistan. Libraries get ample funds from the government for buying books. That money is being spent on new acquisitions, whereas formal assessment to identify the strengths and weaknesses of existing collections is not exercised in a vast majority of cases. It can be safely stated that both awareness and know-how in this regard are very low.
The emerging awareness among the community regarding quality services in public and private sectors, and HEC's vital role in university affairs, demand that collection evaluation be initiated. Libraries are no exception to the rising demand for accountability. Promoting evaluation culture is essential in the developing countries for bringing about a radical shift in the traditional library mindset that perceives, libraries are about collecting, housing, and preserving books for the coming generations. Of course that is true , but the present generation of users is no less important, rather more.
There is a need to turn occasional, informal feedback to a planned, structured collection evaluation program in all types of libraries. Library schools and professional associations must play a significant role in promoting the evaluation culture by making it part of LIS curriculum and designing continuing professional development programs. HEC should als realize the importance of evaluating university libraries not in just through quantitative measures of size, but evaluation of services also.
University librarians in developing countries like Pakistan should consult online resources to sharpen their learning in this regard. Once they are aware of the why and how of formal collection evaluation, they can make use of international practices, adapted to local needs. They must start planning to evolve the evaluation culture voluntarily, before they are forced to do so.
Agee, Jim. 2005. Collection evaluation: a foundation for collection development. Collection Building 24, no. 3 : 92-95.
Ahmad, Nazir. (1984). University library practices in developing countries. London: KPI, Routledge & Kegan Paul.
Ahmad, Nazir. (1984a). Academic libraries in a developing society. Lahore: Qadiria Book Traders.
Ameen, Kanwal (2005). Philosophy and framework of collection management and its application on university library of Pakistan: an appraisal. Unpublished PhD dissertation. University of the Punjab, Lahore.
Ameen, Kanwal (2005a). Developments in the philosophy of collection management: a historical review. Collection Building 24, no 4: 112-116.
Ameen, Kanwal (2008). Issues of book acquisition in university libraries: A case study of Pakistan. Library Philosophy and Practice, http://libr.unl.edu:2000/LPP/ameen.htm. Accessed 10 August 2008.
Ameen, Kanwal and Syed. J. Haider (2007). Book selection strategies in university libraries of Pakistan: An analysis. Library Collections, Acquisitions, and Technical Services 31, no. 3-4: 208–219.
Ameen, Kanwal and Syed. J. Haider. 2007. Evolving paradigm and challenges of collection management in university libraries of Pakistan. Collection Building 26, no. 2: 54-58.
Ameen, Kanwal. March 2008. Barriers in Collection Sharing among Libraries of Pakistan: University library managers' viewpoint. LIBRES 18, no. 1: 1-11. http://libres.curtin.edu.au/libres18n1/. Accessed 13 August 2008.
Bashir, Shaheena. (1991). Measurement of book availability at the Punjab University Library. Unpublished master's thesis. University of the Punjab, Lahore, Pakistan.
Bollen, Johan, Paul Shabajee and Eric Weig. 2002. Evaluation of digital library impact and user communities by analysis of usage patterns. D-Lib Magazine 8, no. 6: http://www.dlib.org/dlib/june02/bollen/06bollen.html. Accessed 10 July, 2008.
Clayton, P., & G. E. Gorman. 2001. Managing information resources in libraries: Collection management in theory and practice. London: Library Association.
Covey, D. T. 2002. Use and usability assessment: library practices and concerns. Washington, DC: Digital Library Federation, Council on Library & Information Resources.
Crawford, J. 2006. The culture of evaluation in LIS. Oxford: Chandos Publishing.
Deshpande, K. S. (1985). University library system in India. New Delhi: Sterling.
Emanuel, Michelle. 2002. Collection evaluation in 150 Hours. Collection Management. Vol. 27, no. ¾: 79-93.
Evans, Edwards G. and M. Z. Saponaro. 2005. Developing library and information centre collections. (5 th ed.). Westport, Connecticut: Libraries Unlimited.
Gorman, G. E. (Ed.). (2000) Collection management: International yearbook of library and information management 2000-2001. London: Library Association.
Haider, S. J. (1993). Acquiring foreign materials for Pakistani libraries: A study. Unpublished doctoral dissertation, Loughborough University, Leicestershire, United Kingdom.
Haider, S.J. (2006). Education on collection management in Pakistan. Collection Management 31, 4: 95-107.
Higher Education Commission (2008) www.hec.gov.pk/new/QualityAssurance/ (accessed September, 2007).
Hiller, Steve, Martha Kyrillidou and Jim Self. (2006). Assessment in North American research libraries: a preliminary report. Performance Measurement and Metrics 7, no. 2: 100-106.
Khalid, H. (1997 ). Cooperation and networking in university libraries: A model for initiation and implementation in countries with less developed systems. Unpublished doctoral dissertation, Manchester Metropolitan University, Manchester, United Kingdom.
Khan, M. Fazil. (1991). Coordinated planning for university libraries in Pakistan: prospects, organization and implementation. Unpublished doctoral dissertation, Islamia University Bahawalpur, Bahawalpur, Pakistan.
Khurshid, Anees. (2000). Planning and management of library and information services in Pakistan. Karachi: Library and Information Services Group.
Lancaster, F. W. (1988). If you want to evaluate your library. Urbana-Champaign: University of Illinois, Graduate School of Library and Information Science.
Lancaster, F. W. (1993). If you want to evaluate your library. 2nd ed. Urbana-Champaign: University of Illinois, Graduate School of Library and Information Science.
Magrill, R. and J. Corbin (1989). Acquisition management and collection development in libraries (2nd ed.). Chicago: American Library Association.
Markless, Sharon and David Streatfield (2006). Evaluating the impact of your library. London: Facet Publishing.
Moid Abdul. (1995). A comparative study of interlibrary loan functions and the development of a model interlibrary loan network among academic libraries in Saudi Arabia. Dissertation Abstracts International, UMI ProQuest Digital Dissertation (UMI No. 9808215)
Moid, Abdul. (1955). Library services in Pakistan: Present conditions and possibilities of further expansion. Submitted as Special Research for Course 392 at the University of Michigan. [Unpublished report].
Mosher, Paul H. (1979). Collection evaluation in research libraries: The search for quality, consistency, and system in collection development. Library Resources and Technical Services 23, no. 1:16-32.
Nisonger, T. E. (1992). Collection Evaluation in Academic Libraries; A Literature Guide and Annotated Bibliography. Englewood, CO: Libraries Unlimited. Also issued as electronic book by netLibrary.
Nisonger, T. E. (2003). Evaluation of library collections, access, and electronic resources: a literature guide and annotated bibliography. Westport, CT: Libraries Unlimited.
Panda, B. (1992). The growth of academic library system. New Delhi: Anmol Publications.
Prasher, R. (2002). India's university libraries: Organization, operation and services. Ludhiana: Medallion Press.
Qureshi, Naimuddin. (1982). Standards for university libraries in Pakistan. Unpublished doctoral dissertation. University of Pittsburgh, Pittsburgh.
Razzaque, M. A. (1971). Academic libraries. In Society for the Promotion and Improvement of Libraries (SPIL). Role of the Library in National Reconstruction. [Proceedings of the seminar held in Rawalpindi, Islamabad in 1967], Karachi: Hamdard National Foundation.
Rout, R. (1992). Operational management of university libraries in India. New Delhi: Reliance Publishing.
Samdani, R., and K. Mahmood (1999). Periodical literature in library and information science: An index of 50 years' work in Pakistan (1947-1997). Karachi: Pakistan Bibliographical Working Group.
Samdani, R., and K. Mahmood, (2003). An on-line index of Pakistan Library Bulletin: Volume 1 to 33 (1968 to 2002). Karachi: Library Promotion Bureau. http://www.geocities.com/plbindex. Accessed 14 August, 2008.
Sharma, R. (1986). Indian academic libraries and Dr. S.R. Ranganathan: A critical study. New Delhi: Sterling.
Sood, S., & Sharma, C. (Eds.). (1991). Outstanding studies in university librarianship. Jaipur: RBSA Publishers.
Tahira, Muzammil (2008). Information needs and seeking behaviors of science and technology teachers of the University of the Punjab. Unpublished MPhil Thesis, University of the Punjab, Lahore, Pakistan.
Williams, D., S. Howell and M. Hricko. 2006. Online assessment, measurement, and evaluation: Emerging practices. Hershey: Information Science Publishing.