Libcitations: A Measure for Comparative Assessment of Book Publications in the Humanities and Social Sciences

White, Howard D. and Boell, Sebastian K. and Yu, Hairong and Davis, Mari and Wilson, Concepción S. and Cole, Fletcher T. H. Libcitations: A Measure for Comparative Assessment of Book Publications in the Humanities and Social Sciences. Journal of the American Society for Information Science and Technology, 2009, vol. 60, n. 6, pp. 1083-1096. [Journal article (Paginated)]

[img]
Preview
PDF
Libcitation-White,Boell,Yu,David,Wilson,Cole_(JASIST_preprint).pdf

Download (217kB) | Preview

English abstract

Bibliometric measures for evaluating research units in the book-oriented humanities and social sciences are underdeveloped relative to those available for journal-oriented science and technology. We therefore present a new measure designed for book-oriented fields: the “libcitation count.” This is a count of the libraries holding a given book, as reported in a national or international union catalog. As librarians decide what to acquire for the audiences they serve, they jointly constitute an instrument for gauging the cultural impact of books. Their decisions are informed by knowledge not only of audiences but also of the book world, e.g., the reputations of authors and the prestige of publishers. From libcitation counts, measures can be derived for comparing research units. Here, we imagine a matchup between the departments of history, philosophy, and political science at the University of New South Wales and the University of Sydney in Australia. We chose the 12 books from each department that had the highest libcitation counts in the Libraries Australia union catalog during 2000–2006. We present each book’s raw libcitation count, its rank within its LC class, and its LC-class normalized libcitation score. The latter is patterned on the item-oriented field normalized citation score used in evaluative bibliometrics. Summary statistics based on these measures allow the departments to be compared for cultural impact. Our work has implications for programs such as Excellence in Research for Australia and the Research Assessment Exercise in the United Kingdom. It also has implications for data mining in OCLC’s WorldCat.

Item type: Journal article (Paginated)
Keywords: Libcitation, Bibliometrics, Humanities, Social Sciences, Library Holdings, Holdings, Australia, LC-Class, Philosophy, History, Political Sciences,
Subjects: D. Libraries as physical collections. > DB. National libraries.
B. Information use and sociology of information
B. Information use and sociology of information > BB. Bibliometric methods
Depositing user: Sebastian K Boell
Date deposited: 16 May 2009
Last modified: 02 Oct 2014 12:14
URI: http://hdl.handle.net/10760/13120

References

Archambault, É., and É. Vignola-Gagne. (2004). The use of bibliometrics in the social sciences and humanities. Science-Metrix final report prepared for the Social Sciences and Humanities Research Council of Canada. Retrieved February 10, 2009, from http://www.sciencemetrix.com/pdf/SM_2004_008_SSHRC_Bibliometrics_Social_Science.pdf

Archambault, É., É. Vignola-Gagne, G. Côté, V. Larivière, Y. Gingras. (2006). Benchmarking scientific output in the social sciences and humanities: The limits of existing databases. Scientometrics, 68, 329-342.

Australian Research Council. (2008). The Excellence in Research for Australia (ERA) Initiative. Retrieved February 10, 2009, from http://www.arc.gov.au/era/

Bensman, S. J. (2007a). Donald J. Urquhart and the integration of science with librarianship: Part 1. Interlending & Document Supply, 35, 74-84.

Bensman, S. J. (2007b). Donald J. Urquhart and the integration of science with librarianship: Part 2. Interlending & Document Supply, 35, 124-130. [See also the papers accessed February 10, 2009, from http://www.garfield.library.upenn.edu/bensman/bensman.html]

Bourke, P., L. Butler and B. Biglia. (1996). Monitoring research in the periphery: Australia and the ISI Indices, Research Evaluation and Policy Project Monograph Series No. 3, Research School of Social Sciences. Canberra: Australian National University.

Butler, L., and M. S. Visser. (2006). Expanding citation analysis to non-source items. Scientometrics, 66, 327-343.

Butler, L., and K. Henadeera. (2007). Is there a role for novel citation measures for the social sciences and humanities in a national research assessment exercise? Proceedings of the 11th International Conference of the International Society for Scientometrics and Informetrics, Madrid, Spain. v. 1: 170-178.

Carr, L., S. Hitchcock, C. Oppenheim J.W. McDonald, T. Champion, and S. Harnad. (2006). Extending journalbased research impact assessment to book-based disciplines (research proposal). Retrieved February 10, 2009, from http://eprints.ecs.soton.ac.uk/12725/

Corbyn, Z. (2008a). Philosophers in ranking protest. Times Higher Education, 28 August 2008. Retrieved February 10, 2009, from http://www.timeshighereducation.co.uk/story.asp?storyCode=403321&sectioncode=26

Corbyn, Z. (2008b). Keep peer review at REF core, chairs warn. Times Higher Education, 28 August 2008. Retrieved February 10, 2009, from http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=403336

Council for the Humanities, Arts, and Social Sciences [CHASS]. (2005). Measures of quality and impact of publicly funded research in the humanities, arts, and social sciences. CHASS Occasional Paper 2. Canberra: Australian National University. Retrieved February 10, 2009, from http://www.chass.org.au/papers/PAP20051101JP.php

Department of Education, Science and Training. (2005). Research Quality Framework: Assessing the quality and impact of research in Australia. Retrieved February 10, 2009, from http://www.dest.gov.au/sectors/research_sector/publications_resources/profiles/rqf_adv_approach.htm

Dolan, C. (2008). Feasibility study: The evaluation and benchmarking of humanities research in Europe. Humanities in the European Research Area report to the UK Arts and Humanities Research Council. Retrieved February 10, 2009, from http://www.heranet.info/Admin/Public/DWSDownload.aspx?File=%2fFiles%2fFiler%2fRapporter+HERA%2fRapporter+jan+08%2fHERA_Study_final.pdf

Donovan, C. (2005). A review of current Australian and international practice in measuring the quality and impact of publicly funded research in the humanities, arts, and social sciences. Retrieved February 10, 2009 from http://repp.anu.edu.au/papers/200511_disc03_review.pdf

Donovan, C. (2007). The hidden perils of citation counting for Australasian political science. Australian Journal of Political Science, 42, 665-678.

Donovan, C., and L. Butler. (2007). Testing novel quantitative indicators of research ‘quality’, esteem and ‘user engagement’: An economics pilot study. Research Evaluation, 16, 231-242.

Drummond, R. (2007). RQF information session: Measuring research. Presentation at the University of New South Wales Library. Retrieved February 10, 2009, from http://info.library.unsw.edu.au/libadmin/pdf/unswrqfsession.pdf

East, J. W. (2006) Ranking journals in the humanities: An Australian case study. Australian Academic and Research Libraries, 37, 3-16.

Evidence Ltd. (2007). The use of bibliometrics to measure research quality in UK higher education institutions. London: Universities UK. Retrieved February 10, 2009, from http://www.universitiesuk.ac.uk/Publications/Documents/bibliometrics.pdf

Goodson, L. P., B. Dillman, and A. Hira. (1999). Ranking the presses: Political scientists’ evaluations of publisher quality. PS : Political Science & Politics, 32, 257-262.

Haddow, G. (2008) Quality Australian journals in the humanities and social sciences. Australian Academic and Research Libraries, 39, 79-91.

Holmes, A., and C. Oppenheim. (2001). Use of citation analysis to predict the outcome of the 2001 Research Assessment Exercise for Unit of Assessment (UoA) 61: Library and Information Management. Information Research 6(2). Retrieved February 10, 2009, from http://informationr.net/ir/6-2/paper103.html

Lewis, J. S. (2000). An assessment of publisher quality by political science librarians. College and Research Libraries, 61, 313-323.

Lundberg, J. (2007). Lifting the crown – Citation z score. Journal of Informetrics,1, 145–154.

Metz, P., and J. Stemmer. (1996). A reputational study of academic publishers. College and Research Libraries, 57, 234-247.

Mingers, J., and A.W.K. Harzing. (2007) Ranking journals in Business and Management: A statistical analysis of the Harzing Dataset. European Journal of Information Systems, 16, 303-316.

Missingham, R., and R. Walls. (2003). Australian university libraries: Collections overlap study. Australian Library Journal, 52, 247-260. Retrieved February 10, 2009, from http://www.alia.org.au/publishing/alj/52.3/full.text/missingham.walls.html Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht, Netherlands: Springer Verlag.

Moed., H. F., M. Luwel, and A. J. Nederhof. (2002). Toward research performance in the humanities. Library Trends, 50, 498-520.

Najman, J. M., and B. Hewitt. (2003). The validity of publication and citation counts for Sociology and other selected disciplines. Journal of Sociology, 39, 62-80.

National Library of Australia. (2008). The Australian National Bibliographic Database. Retrieved February 10, 2009, from http://www.nla.gov.au/libraries/resource/nbd.html

Nederhof, A. J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities. Scientometrics, 66, 81-100.

Nederhof, A. J., and E. C. M. Noyons. (1992). International comparison of departments’ research performance in the humanities. Journal of the American Society for Information Science, 43, 249-256.

Norris, M., and C. Oppenheim. (2007). Comparing alternatives to the Web of Science for coverage of the social sciences’ literature. Journal of Informetrics, 1, 161-169.

Oppenheim, C. (1997). The correlation between citation counts and the 1992 Research Assessment Exercise ratings for British research in genetics, anatomy and archaeology. Journal of Documentation, 53, 477-487.

Oppenheim, C., and M. Norris. (2003). Citation counts and the research assessment exercise V: Archaeology and the 2001 RAE. Journal of Documentation, 59, 709-30.

Oppenheim, C., and M. A. C. Summers. (2008). Citation counts and the Research Assessment Exercise, part VI: Unit of assessment 67 (music). Information Research 13(2). Retrieved February 10, 2009, from http://informationr.net/ir/13-2/paper342.html

Phelan, T. J. (1999). A compendium of issues for citation analysis. Scientometrics, 45, 117-136.

Rehn, C., Kronman, U., & Wadskog, D. (2007) Bibliometric indicators - Definitions and usage at Karolinska Institutet. Appendix to Bibliometric Handbook for Karolinska Institutet. Stockholm, Sweden: Karolinska Institutet University Library. Retrieved February 10, 2009, from http://ki.se/content/1/c6/01/79/31/Bibliometric%20indicators%20-%20definitions_1.0.pdf

Research Evaluation and Policy Project [REPP]. (2005). Quantitative indicators for research assessment—A literature review. REPP Discussion Paper 05/1. Research School of Social Sciences. Canberra: Australian National University. Retrieved February 10, 2009, from http://repp.anu.edu.au/workingpapers.php

Sandström, U., and E. Sandstrom. (2007). A metric for academic performance applied to Australian universities 2001-2004. Retrieved February 10, 2009, from http://eprints.rclis.org/archive/00011776/01/2metrics.pdf

Seng, L. B., and P. Willett. (1995). The citedness of publications by United Kingdom library schools. Journal of Information Science, 21, 68-71.

Smith, A. T., and M. Eysenck. (2002). The correlation between RAE ratings and citation counts in psychology. Technical Report, Department of Psychology, Royal Holloway, University of London. Retrieved February 10, 2009, from http://cogprints.org/2749/

Torres-Salinas, D., and H. F. Moed. (2008). Library catalog analysis is a useful tool in studies of social sciences and humanities. In Excellence and Emergence. A New Challenge for the Combination of Quantitative and Qualitative Approaches. 10th International Conference on Science and Technology Indicators. Vienna, Austria. 246-250.

Use of research metrics in the arts and humanities: Report of the expert group set up jointly by the Arts and Humanities Research Council and the Higher Education Funding Council for England. (2006). Retrieved February 10, 2009, from http://www.hefce.ac.uk/research/ref/group/metrics.pdf

White, H. D. (1995). Brief tests of collection strength: A methodology for all types of libraries. Westport, CT: Greenwood Press.

White, H. D. (2008). Better than brief tests: Coverage power tests of collection strength. College and Research Libraries, 69, 155-174.

Wiberley, S. E. (2002). The humanities: Who won the '90s in scholarly book publishing. Libraries and the Academy 2, 357-374

Wiberley, S. E. (2004). The social sciences: Who won the '90s in scholarly book publishing. College and Research Libraries, 65, 505-523.

Wright, M., and J. S. Armstrong. (2008). The ombudsman: Verification of citations: Fawlty Towers of knowledge? Interfaces, 38, 125-132.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item