Journal Impact Factors for evaluating scientific performance : use of h-like indicators

Boell, Sebastian K. and Wilson, Concepción S. Journal Impact Factors for evaluating scientific performance : use of h-like indicators. Scientometrics, 2010, vol. 82, n. 3, pp. 613-626. [Journal article (Paginated)]

[img]
Preview
 PDF
2010_SCIENTOMETRICS-Boell,Wilson-h-like-postprint.pdf
Download (327kB) | Preview

English abstract

This article introduces the Impact Factor squared or IF²-index, an h-like indicator of research performance. This indicator reflects the degree to which large entities such as countries and/or their states participate in top-level research in a field or subfield. The IF²-index uses the Journal Impact Factor (JIF) of research publications instead of the number of citations. This concept is applied to other h-type indexes and their results compared to the IF²-index. These JIF-based indexes are then used to assess the overall performance of cancer research in Australia and its states over 8 years from 1999 to 2006. The IF²-index has three advantages when evaluating larger research units: firstly, it provides a stable value that does not change over time, reflecting the degree to which a research unit participated in top-level research in a given year; secondly, it can be calculated closely approximating the publication date of yearly datasets; and finally, it provides an additional dimension when a full article-based citation analysis is not feasible. As the index reflects the degree of participation in top-level research it may favor larger units when units of different sizes are compared.

Item type: Journal article (Paginated)
Keywords: research evaluation Journal impact factor IF²-index Australia cancer research h-index h-type indexes h-like indexes Scientometrics
Subjects: B. Information use and sociology of information > BB. Bibliometric methods
Depositing user: Sebastian K Boell
Date deposited: 10 Apr 2010
Last modified: 02 Oct 2014 12:16
URI: http://hdl.handle.net/10760/14438

References

Arencibia-Jorge, R., Barrios-Almaguer, I., Ferna´ndez-Herna´ndez, S., & Carvajal-Espino, R. (2008). Applying successive H indices in the institutional evaluation: A case study. Journal of the American Society for Information Science and Technology, 59(1), 155–157.

Bensman, J. S. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41, 93–155

Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381–1385.

Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830–837.

Bornmann, L., Wallon, G., & Ledin, A. (2008b). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation, 17(2), 149–156.

Braun, T., Glänzel, W., & Schubert, A. (2005). A Hirsch-type index for journals. The Scientist, 19(22), 8.

Braun, T., Glänzel, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 167–173.

Burrell, Q. L. (2007a). Should the h-index be discounted? ISSI Newsletter, 3(S), 65–68.

Burrell, Q. L. (2007b). On the h-index, the size of the Hirsch core and Jin’s A-index. Journal of Informetrics, 1(2), 170–177.

Burrell, Q. L. (2009). On Hirsch’s h, Egghe’s g and Kosmulski’s h(2). Scientometrics. doi: 10.1007/ s11192-009-0405-3 [Online First].

Costas, R., & Bordons, M. (2007). The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro-level. Journal of Informetrics, 1(3), 193–203.

Costas, R., & Bordons, M. (2008). Is g-index better than h-index? An exploratory study at the individual level. Scientometrics, 77(2), 267–288.

Cronin, B., & Meho, L. (2006). Using the h-index to rank influential information scientists. Journal of the American Society for Information Science and Technology, 57(9), 1275–1278.

Davies, H., Bignell, G. R., Cox, C., Stephens, P., Edkins, S., Clegg, S., et al. (2002). Mutations of the BRAF gene in human cancer. Nature, 417(6892), 949–954.

Egghe, L. (2006). Properties of the n-overlap vector and n-overlap similarity theory. Journal of the American Society for Information Science and Technology, 57(9), 1165–1177.

Egghe, L. (2007). Dynamic h-index: The Hirsch index in function of time. Journal of the American Society for Information Science and Technology, 58(3), 452–454.

Egghe, L. (2008). The influence of merging on h-type indices. Journal of Informetrics, 2(3), 252–262.

Egghe, L. (2010). The Hirsch index and related impact measures. Annual Review of Information Science and Technology, 44, 65–114.

Geyer, C. E., Forster, J., Lindquist, D., Chan, S., Romieu, C. G., Pienkowski, T., et al. (2006). Lapatinib plus capecitabine for HER2-positive advanced breast cancer. The New England Journal of Medicine, 355(26), 2733–2743.

Glänzel, W. (2008). On some new bibliometric applications of statistics related to the h-index. Scientometrics, 77(1), 187–196.

Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

Jin, B. (2006). h-index: An evaluation indicator proposed by scientist. Science Focus, 1(1), 8–9.

Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and ARindices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855–863.

Kaltenborn, K.-F., & Kuhn, K. (2004). The journal impact factor as a parameter for the evaluation of researchers and research. Revista Espan˜ola de Enfermedades Digestivas, 96(7), 460–476. http://scielo. isciii.es/scielo.php?script=sci_arttext&pid=S1130-01082004000700004&lng=en&nrm=iso [Online].

Ketcham, C. M., & Crawford, J. M. (2008). Can impact factor data be trusted? Laboratory Investigation, 88, 340–341.

Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

Lee, K. P., Schotland, M., Bacchetti, P., & Bero, L. A. (2002). Association of journal quality indicators with methodological quality of clinical research articles. The Journal of the American Medical Association, 287(21), 2805–2808.

Lehrl, S. (1999). Der Impact-Faktor als Bewertungskriterium wissenschaftlicher Leistungen- das Recht auf Chancengleichheit. Strahlentherapie und Onkologie, 175(4), 141–153.

Molinari, J.-F., & Molinari, A. (2008). A new methodology for ranking scientific institutions. Scientometrics, 75(1), 163–174.

Oppenheim, C. (2007). Using the h-index to rank influential British researchers in information science and librarianship. Journal of the American Society for Information Science and Technology, 58(2), 297–301.

Opthof, T. (1997). Sense and nonsense about the impact factor. Cardiovascular Research, 33, 1–7.

Pires Da Luz, M., Marques-Portella, C., Mendlowicz, M., Gleiser, S., Silva Freire Coutinho, E., & Figueira, I. (2008). Institutional h-index: The performance of a new metric in the evaluation of Brazilian psychiatric post-graduation programs. Scientometrics, 77(2), 361–368.

Pringle, J. (2008). Trends in the use of ISI citation databases for evaluation. Learned Publishing, 21(2), 85–91.

Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the data. Journal of Cell Biology, 179(6), 1091– 1092.

Rossner, M., Van Epps, H., & Hill, E. (2008). Irreproducible results: A response to Thomson Scientific. Journal of Cell Biology, 180(2), 254–255.

Rousseau, R. (2006). New developments related to the Hirsch index. Accessed October 14, 2008, from http://eprints.rclis.org/archive/00006376/01/Hirsch_new_developments.pdf [Online].

Rousseau, R. (2008). Reflections on recent developments of the h-index and h-type indices. In H. Kretschmer & F. Havemann (Eds.), Proceedings of WIS 2008. Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. Berlin: HumboldtUniversität zu Berlin, Institute for Library and Information Science. http://www.collnet.de/Berlin-2008/RousseauWIS2008rrd. pdf [Online].

Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.

Schreiber, M. (2008). An empirical investigation of the g-index for 26 physicists in comparison with the h-Index, the Aindex, and the R-index. Journal of the American Society for Information Science and Technology, 59(9), 1513–1522.

Schubert, A. (2009). Using the h-index for assessing single publications. Scientometrics, 78(3), 559–565.

Schubert, A., & Glänzel, W. (2007). A systematic analysis of Hirsch-type indices for journals. Journal of Informetrics, 1(3), 179–184.

Schubert, A., Korn, A., & Telcs, A. (2009). Hirsch-type indices for characterizing network. Scientometrics, 78(2), 375–382.

Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45(1), 1–11.

Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491–502.

Vanclay, J. K. (2007). On the robustness of the h-index. Journal of the American Society for Information Science and Technology, 58(10), 1547–1550.

Vanclay, J. K. (2008). Ranking forestry journals using the h-index. Journal of Informetrics, 2(4), 326–334.

Welberry, H., Edwards, C., Weston, A., Harvey, C., Wilson, C. S., Boell, S. K., Lo, M., & Bishop, J. K. (2008). Cancer research in New South Wales 2001–2006. Sydney: Cancer Institute NSW. http://www.cancerinstitute.org.au/cancer_inst/publications/pdfs/rm-2008-1_cancer-research-in-nsw-2001-2006.pdf [Online].

Wilson, C. S. (2005). General analyses of cancer research publications in Australian states using the science and social science citations indexes. In P. Ingwersen & B. Larsen (Eds.), Proceedings of ISSI 2005 (pp. 168–176). Stockholm: Karolinska University Press.

Wilson, C. S., Boell, S. K., & Lo, M. (2007). Systematic review of the quantity and quality of cancer research publications in New South Wales: 1999 to 2006. Sydney: University of New South Wales, School of Information Systems, Technology and Management.

Wilson, C. S., & Pittman, S. (2000). Assessments of outputs: Quantity and quality of cancer research publications in New South Wales from 1994 to 1998. Sydney: The University of New South Wales, School of Information Systems, Technology and Management.

Yue, W., Wilson, C. S., & Boller, F. (2007). Peer assessment of journal quality in clinical neurology. Journal of the Medical Library Association, 95(1), 70–76.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS