Impact of Data Sources on Citation Counts and Rankings of LIS Faculty: Web of Science vs. Scopus and Google Scholar

Meho, Lokman I. and Yang, Kiduk Impact of Data Sources on Citation Counts and Rankings of LIS Faculty: Web of Science vs. Scopus and Google Scholar., 2007 [Preprint]

[img]
Preview
 PDF
meho-yang-03.pdf
Download (624kB) | Preview

English abstract

The Institute for Scientific Information's (ISI) citation databases have been used for decades as a starting point and often as the only tools for locating citations and/or conducting citation analyses. ISI databases (or Web of Science [WoS]), however, may no longer be sufficient because new databases and tools that allow citation searching are now available. Using citations to the work of 25 library and information science faculty members as a case study, this paper examines the effects of using Scopus and Google Scholar (GS) on the citation counts and rankings of scholars as measured by WoS. Overall, more than 10,000 citing and purportedly citing documents were examined. Results show that Scopus significantly alters the relative ranking of those scholars that appear in the middle of the rankings and that GS stands out in its coverage of conference proceedings as well as international, non-English language journals. The use of Scopus and GS, in addition to WoS, helps reveal a more accurate and comprehensive picture of the scholarly impact of authors. WoS data took about 100 hours of collecting and processing time, Scopus consumed 200 hours, and GS a grueling 3,000 hours.

Item type: Preprint
Keywords: Web of Science, ISI Databases, Scopus, Google Scholar, Citation Analysis, Bibliometrics, Webometrics, Web Metrics, Scientometrics, Informetrics, Databases, Database Evaluation, Research Evaluation, Tenure and Promotion, Scholarly Communication
Subjects: H. Information sources, supports, channels. > HL. Databases and database Networking.
B. Information use and sociology of information > BB. Bibliometric methods
Depositing user: Lokman I. Meho
Date deposited: 26 Jan 2007
Last modified: 02 Oct 2014 12:06
URI: http://hdl.handle.net/10760/8876

References

Adkins, D., & Budd, J. (2006). Scholarly productivity of U.S. LIS faculty. Library & Information Science Research, 28(3), 374-389.

Aksnes, D. W., & Taxt, R. E. (2004). Peer reviews and bibliometric indicators: A comparative study at a Norwegian university. Research Evaluation, 13(1), 33-41.

Association for Library and Information Science Education. (2006). LIS research areas classification scheme. Retrieved March 15, 2006, from http://www.alise.org/research_class_guide.html.

Bakkalbasi, N., Bauer, K., Glover, J., & Wang, L. (2006). Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries, 7. Retrieved October 04, 2006 from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1533854.

Ballard, S., & Henry, M. (2006). Citation searching: New players, new tools. The Searcher: the Magazine for Database Professionals, 14(9), 24-33.

Bar-Ilan, J. (2006). H-index for Price medalists revisited. ISSI Newsletter, 2(1), 3-4.

Bauer, K., & Bakkalbasi, N. (2005). An examination of citation counts in a new scholarly communication environment. D-Lib Magazine, 11(9). Retrieved March 25, 2006, from http://www.dlib.org/dlib/september05/bauer/09bauer.html.

Borgman, C. L., & Furner, J. (2002). Scholarly communication and bibliometrics. Annual Review of Information Science and Technology, 36, 3-72.

Börner, K., Chen, C., & Boyack, K. W. (2003). Visualizing knowledge domains. Annual Review of Information Science and Technology, 37, 179-255.

Börner, K., Sanyal, S., & Vespignani, A. (2006). Network Science: A Theoretical and Practical Framework. Annual Review of Information Science and Technology, 40.

Bornmann, L., & Daniel, H-D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology.

Bornmann, L., & Daniel, H-D. (2005). Does the h-index for ranking of scientists really work? Scientometrics, 65(3), 391-392.

Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169–173.

Budd, J. M. (2000). Scholarly productivity of U.S. LIS faculty: An update. The Library Quarterly, 70(2), 230-245.

Cronin, B. (1984). The citation process: The role and significance of citations in scientific communication. London: Taylor Graham.

Cronin, B., & Meho, L. (2006). Using the h-index to rank influential information scientists. Journal of the American Society for Information Science and Technology, 57(9), 1275-1278.

Cronin, B., Snyder, H., & Atkins, H. (1997). Comparative citation rankings of authors in monographic and journal literature: A study of sociology. Journal of Documentation, 53(3), 263-273.

de Arenas, J. L., Castanos-Lomnitz, H., & Arenas-Licea, J. (2002). Significant Mexican research in the health sciences: A bibliometric analysis. Scientometrics, 53(1), 39-48.

Elsevier Science Publishers (2006). Scopus. New York: Elsevier. [Electronic database]

Funkhouser, E. T. (1996). The evaluative use of citation analysis for communications journals. Human Communication Research, 22(4), 563-574.

Gardner, S., & Eng, S. (2005). Gaga over Google? Scholar in the Social Sciences. Library Hi Tech News, 22(8), 42-45.

Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA-Journal of the American Medical Association, 295(1), 90-93.

Garfield, E. (1996). How can impact factors be improved? British Medical Journal, 313 (7054), 411-413.

Glänzel, W. (1996). The needs for standards in bibliometric research and technology. Scientometrics, 35(2), 167-176.

Goodman, D., & Deis, L. (2005). Web of Science (2004 version) and Scopus. The Charleston Advisor, 6(3). Retrieved March 25, 2006, from http://www.charlestonco.com/dnloads/v6n3.pdf.

Goodrum, A. A., McCain, K. W., Lawrence, S., & Giles, C. L. (2001). Scholarly publishing in the Internet age: A citation analysis of computer science literature. Information Processing & Management, 37(5), 661-675.

Harnad, S., & Brody, T. (2004). Comparing the impact of open access (OA) vs. non-OA articles in the same journals. D-Lib Magazine, 10(6). Retrieved on April 27, 2006 from http://www.dlib.org/dlib/june04/harnad/06harnad.html.

Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569-16572. Retrieved February 15, 2006, from http://www.pnas.org/cgi/reprint/102/46/16569.

Holden, G., Rosenberg, G., & Barker, K. (2005). Bibliometrics: A potential decision making aid in hiring, reappointment, tenure and promotion decisions. Social Work in Health Care, 41(3-4), 67-92.

Jacsó, P. (2006). Deflated, inflated and phantom citation counts. Online Information Review, 30(3), 297-309.

Jacsó, P. (2005a). As we may search—comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Current Science, 89(9), 1537-1547. Retrieved March 15, 2006, from http://www.ias.ac.in/currsci/nov102005/1537.pdf.

Jacsó, P. (2005b). Google Scholar: The pros and the cons. Online Information Review, 29(2), 208-214.

Kelly, C. D., & Jennions, M. D. (2006). The h index and career assessment by numbers. TRENDS in Ecology and Evolution, 21(4), 167-170.

Kostoff, R. N. (1996). Performance measures for government-sponsored research: Overview and background. Scientometrics, 36(3), 281-292.

MacRoberts, M. H., & MacRoberts, B. R. (1996). Problems of citation analysis. Scientometrics, 36(3), 435-444.

Martin, B. R. (1996). The use of multiple indicators in the assessment of basic research. Scientometrics, 36(3), 343-362.

Meho, L. I., & Sonnenwald, D. H. (2000). Citation ranking versus peer evaluation of senior faculty research performance: A case study of Kurdish scholarship. Journal of the American Society for Information Science, 51(2), 123-138.

Merton, R. K. (1968). The Matthew Effect in science. Science, 159(3810), 56-63.

Moed, H. F. (2005), Citation analysis in research evaluation, Springer, Berlin.

Narin, F. (1976). Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity. Cherry Hill, NJ: Computer Horizons.

Nisonger, T. E. (2004a). Citation autobiography: An investigation of ISI database coverage in determining author citedness. College & Research Libraries, 65(2), 152-163.

Nisonger, T. E. (2004b). The benefits and drawbacks of impact factor for journal collection management in libraries. The Serials Librarian, 47(1-2), 57-75.

Nisonger, T. E., & Davis, C. H. (2005). The perception of library and information science journals by LIS education deans and ARL library directors: A replication of the Kohl-Davis study. College & Research Libraries, 66(4), 341-377.

Noruzi, A. (2005). Google Scholar: The new generation of citation indexes. Libri, 55(4), 170-180.

Oppenheim, C. (2007). Using the h-index to rank influential British researchers in information science and librarianship. Journal of the American Society for Information Science and Technology, 58(2).

Pauly, D., & Stergiou, K. I. (2005). Equivalence of results from two citation analyses: Thomson ISI’s citation index and Google’s scholar service. Ethics in Science and Environmental Politics, 33-35.

Persson, O., & Åström, F. (2005). Most cited universities and authors in Library & Information Science 1990-2004. Bibliometric Notes, 7(2). Retrieved November 15, 2006, from http://www.umu.se/inforsk/BibliometricNotes/BN2-2005/BN2-2005.htm.

Reed, K. L. (1995). Citation analysis of faculty publications: Beyond Science Citation Index and Social Science [sic] Citation Index. Bulletin of the Medical Library Association, 83(4), 503-508.

Roth, D. L. (2005). The emergence of competitors to the Science Citation Index and Web of Science. Current Science, 89(9), 1531-1536. Retrieved March 15, 2006, from http://www.ias.ac.in/currsci/nov102005/1531.pdf.

Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42-46.

Seglen, P. O. (1998). Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica Scandinavica, 69(3), 224-229.

Small, H. (1999). Visualizing science by citation mapping. Journal of the American Society for Information Science, 50(9), 799-813.

So, C. Y. K. (1998). Citation ranking versus expert judgment in evaluating communication scholars: Effects of research specialty size and individual prominence. Scientometrics, 41(3), 325-333.

Thomson Corporation. (2006a). Web of Science 7.0. Retrieved June 15, 2005, from http://scientific.thomson.com/support/products/wos7/.

Thomson Corporation. (2006b). University science indicators, 1981-2005. in-cites. Retrieved May 10, 2006, from http://in-cites.com/research/2006/may_8_2006-1.html.

Thomson Scientific. (2006). Journal Citation Reports. Philadelphia: Institute for Scientific Information. [Electronic resource]

van Raan, A. F. J. (1996). Advanced bibliometric methods as quantitative core of peer-review based evaluation and foresight exercises. Scientometrics, 36(3), 397-420.

van Raan, A. F. J. (2000). The pandora’s box of citation analysis: Measuring scientific excellence—the last evil? In B. Cronin & H. B. Atkins (Eds.). The Web of knowledge: A festschrift in honor of Eugene Garfield (pp. 301-319). Medford, NJ: Information Today.

van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133-143.

Wallin, J. A. (2005). Bibliometric methods: Pitfalls and Possibilities. Basic & Clinical Pharmacology & Toxicology, 97(5), 261-275.

White, H. D., & McCain, K. W. (1997). Visualization of Literatures. Annual Review of Information Science and Technology, 32, 99-168.

Whitley, K. M. (2002). Analysis of SciFinder Scholar and Web of Science citation searches. Journal of the American Society for Information Science and Technology, 53(14), 1210-1215.

Wleklinski, J. M. (2005). Studying Google Scholar: Wall to wall coverage? Online, 29(3), 22-26.

Zhao, D. Z., & Logan, E. (2002). Citation analysis using scientific publications on the web as data source: A case study in the XML research area. Scientometrics, 54(3), 449-472.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS