Productivity and impact of Spanish researchers: reference thresholds within scientific areas

Jiménez-Contreras, Evaristo, Robinson-García, Nicolás and Cabezas-Clavijo, Álvaro Productivity and impact of Spanish researchers: reference thresholds within scientific areas. Revista española de documentación científica, 2011, vol. 34, n. 4. [Journal article (Unpaginated)]

[thumbnail of Productivity and impact of Spanish researchers.pdf]
Preview
 PDF
Productivity and impact of Spanish researchers.pdf
Download (311kB) | Preview

English abstract

Reference thresholds for the scientific production and impact of internationally visible Spanish research within the areas defined by the Spanish National Agency for Evaluation and Prospective (ANEP) are presented. These percentile reference tables are constructed from the population of researchers who applied for a project within the 2007 National R & D Plan (n = 3.356) and are to be used as benchmarks, permitting comparisons between researchers’ bibliometric behavior and mean performance in their respective scientific disciplines. Data relating to mean production, impact and visibility for each ANEP area is also presented. The internationalization process of these areas between 2000 and 2006 is discussed, with special emphasis on the Social Sciences. Finally, we recommend these reference thresholds be used by funding agencies and research institutions as instruments of evaluation in their selection processes

Spanish abstract

Se presentan umbrales de referencia de producción e impacto científico de la investigación española con visibilidad internacional para las áreas defnidas por la Agencia Nacional de Evaluación y Prospectiva (ANEP) en sus convocatorias. Tomando como población los solicitantes de proyectos del Plan Nacional de I+D 2007 (n = 3.356) se construyen tablas de referencia por percentiles que funcionan a modo de benchmarks, permitiendo efectuar comparaciones entre el comportamiento bibliométrico de un investigador y los registros de referencia en su área científica. Igualmente se ofrecen los datos de producción, impacto y visibilidad promedios para las áreas ANEP, y se discute el proceso de internacionalización de dichas áreas en el período 2000-2006, con una especial atención a las Ciencias Sociales. Finalmente, se sugiere el uso de umbrales de referencia como método de evaluación tanto para agencias financiadoras como para instituciones de investigación en sus procesos selectivos

Item type: Journal article (Unpaginated)
Keywords: Bibliometric indicators, reference thresholds, benchmarking, ANEP, science, scientific research, Spain
Subjects: B. Information use and sociology of information
B. Information use and sociology of information > BB. Bibliometric methods
Depositing user: Álvaro Cabezas Clavijo
Date deposited: 02 Apr 2012
Last modified: 02 Oct 2014 12:21
URI: http://hdl.handle.net/10760/16784

References

Abramo, G., and D’Angelo, C. A. (2011). National-scale research performance assessment at the individual level. Scientometrics, vol. 86 (2), 347-364. http://dx.doi.org/10.1007/s11192-010-0297-2

Alonso-Arroyo, A.; Pulgarín, A., and Gil-Leiva, I. (2006). Análisis bibliométrico de la producción científica de la Universidad Politécnica de Valencia 1973-2001. Revista Española de Documentación Científica, vol. 29 (3), 345-363.

Bordons, M.; Fernández, M., and Gómez, I. (2002). Advantages and limitations in the use of impact factor measures for the assessment of research performance. Scientometrics, vol. 53 (2), 195-206. http://dx.doi.org/10.1023/A:1014800407876

Buela-Casal, G.; Bermúdez, M. P.; Sierra, J. C.; Quevedo-Blasco, R., and Castro, Á. (2010). Ranking de 2009 en investigación de las universidades públicas españolas. Psicothema, vol. 22, 171-179. PMid:20423618

Buela-Casal, G.; Medina, A.; Viedma, M.; Godoy, V.; Lozano, S., and Torres, G. (2004). Factor de impacto de tres revistas españolas de Psicología. Psicothema, vol. 16 (4), 680-688.

Butler, L. (2003). Explaining Australia’s increased share of ISI publications – the effects of a funding formula based on publication counts. Research Policy, vol. 32 (1), 143-155. CNEAI. (2005). Memoria sobre la situación del profesorado numerario con respecto a los sexenios de investigación. Año 2005.

Camí, J. (1997). Impactolatría: diagnóstico y tratamiento. Medicina Clínica, vol. 109 (13), 515-524.

Carpenter, M. P.; Gibb, F.; Harris, M.; Irvine, J.; Martin, B. R., and Narin, F. (1988). Bibliometric profi les for British academic institutions: An experiment to develop research output indicators. Scientometrics, vol. 14 (3-4), 213-233. http://dx.doi.org/10.1007/BF02020076

Costas, R. (2008). Análisis bibliométrico de la actividad científi ca de los investigadores del CSIC en tres áreas: Biología y Biomedicina, Ciencia de Materiales y Recursos Naturales. Una aproximación metodológica a nivel micro (Web of Science, 1994-2004). [Tesis doctoral] Universidad Carlos III.

Costas, R., and Bordons, M. (2005). Bibliometric indicators at the micro-level: some results in the area of natural resources at the Spanish CSIC. Research Evaluation, vol. 14 (2), 110-120. http://dx.doi.org/10.3152/147154405781776238

Costas, R.; van Leeuwen, T. N., and Bordons, M. (2010). A bibliometric classifi catory approach for the study and assessment of research performance at the individual level: the effects of age on productivity and impact. Journal of the American Society for Information Science and Technology, vol. 61 (8), 1564–1581.

Cronin, B., and Meho, L. (2006). Using the h-index to rank influential information scientists. Journal of the American Society for Information Science and Technology, vol. 57 (9), 1275-1278. http://dx.doi.org/10.1002/asi.20354

Cruz-Castro, L., and Sanz-Menéndez, L. (2007). Research Evaluation in transition: individual versus organisational assessment in Spain. En: Whitley, R., Gläser, J. (editors). The Sociology of Sciences Yearbook: The Changing Governance of the Sciences. The Advent of the Research Evaluation Systems. Springer, Dordrecht.

Delgado-López-Cózar, E.; Jiménez-Contreras, E., and Ruiz-Pérez, R. (2009). España y los 25 grandes de la ciencia mundial en cifras (1992-2008). El Profesional de la Información, vol. 18 (1), 81-86. http://dx.doi.org/10.3145/epi.2009.ene.11

Garfield, E. (2003). The meaning of the Impact Factor. International Journal of Clinical and Health Psychology, vol. 3 (2), 363-369.

González-De-Dios, J.; Valderrama-Zurian, J. C.; Gonzalez-Alcaide, G.; Sempere, A. P.; Bolanos-Pizarro, M., and Aleixandre-Benavent, R. (2009). Approximation to the impact of biomedical journals in neurological sciences: study of bibliometric indicators in the Journal Citation Reports-Science Citation Index 2006. Revista de Neurología, vol. 48 (3), 117-128. PMid:19206058

Gordillo, V.; González, J., and Muñiz, J. (2004). La evaluación de Proyectos de Investigación por la Agencia Nacional de Evaluación y Prospectiva. Psicothema, vol. 16 (3), 343-349.

Jiménez-Contreras, E.; Delgado-López-Cózar, E., and Ruiz-Pérez, R. (2006). Producción española en biblioteconomía y documentación con visibilidad internacional a través del Web of Science (1995-2004). El Profesional de la Información, vol. 15 (5), 373-383.

Jiménez-Contreras, E.; Moya-Anegón, F. D., and Delgado-López-Cózar, E. (2003). The evolution of research activity in Spain: The impact of the National Commission for the Evaluation of Research Activity (CNEAI). Research Policy, vol. 32 (1), 123-142.

Lawrence, P. (2002). Rank injustice. Nature, vol. 415 (6874), 835-836. http://dx.doi.org/10.1038/415835a PMid:11859339

Lewison, G.; Cottrell, R., and Dixon, D. (1999). Bibliometric indicators to assist the peer review process in grant decisions. Research Evaluation, vol. 8 (1), 47-52. http://dx.doi.org/10.3152/147154499781777621

Macías-Chapula, C.; Rodea-Castro, I.; Gutierrez-Carrasco, A., and Mendoza-Guerrero, J. (2004). Producción científica institucional y posicionamiento nacional: el caso del Hospital General de México. Revista Española de Documentación Científica, vol. 27 (4), 482-497.

Meho, L. I., and Spurgin, K. M. (2005). Ranking the research productivity of library and information science faculty and schools: An evaluation of data sources and research methods. Journal of the American Society for Information Science and Technology, vol. 56 (12), 1314-1331. http://dx.doi.org/10.1002/asi.20227

Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht; Springer. Moed, H.; Burger, W.; Frankfort, J., and van Raan, A. F. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, vol. 14 (3), 131-149. http://dx.doi.org/10.1016/0048-7333(85)90012-5

Moya Anegón, F.; Chinchilla Rodríguez, Z.; Corera Álvarez, E.; Vargas Quesada, B.; Muñoz Fernández, F., and Herrero Solana, V. (2005). Análisis de dominio institucional: la producción científica de la Universidad de Granada (SCI 1991-99). Revista Española de Documentación Científica, vol. 28 (2), 170-195.

Oppenheim, C. (2007). Using the h-index to rank influential British researchers in information science and librarianship. Journal of the American Society for Information Science and Technology, vol. 58 (2), 297-301. http://dx.doi.org/10.1002/asi.20460

Ruiz-Pérez, R.; Delgado-López-Cózar, E., and Jiménez-Contreras, E. (2002). Spanish name variations in national and international biomedical databases: implications for information retrieval and bibliometric studies. Journal of the Medical Library Association, vol. 90 (4), 411-430. PMid:12398248 PMCid:128958

Ruiz-Pérez, R.; Jiménez-Contreras, E., and Delgado-López-Cózar, E. (2008). Complementos bibliométricos de Thomson Scientific en la Web: buenos, bonitos y gratuitos. El Profesional de la Información, vol. 17 (5), 559-563. http://dx.doi.org/10.3145/epi.2008.sep.11

Sanz-Menéndez, L. (1997). Estado, ciencia y tecnología en España: 1939-1997. Madrid; Alianza.

Schubert, A., and Braun, T. (1996). Cross-fi eld normalization of scientometric indicators. Scientometrics, vol. 36 (3), 311-324. http://dx.doi.org/10.1007/BF02129597

Seglen, P. (1997). Why the impact factor of journals should not be used for evaluating research. British Medical Journal, vol. 314 (7079), 498-502. http://dx.doi.org/10.1136/bmj.314.7079.497

Thijs, B., and Glänzel, W. (2009). A structural analysis of benchmarks on different bibliometrical indicators for European research institutes based on their research profile. Scientometrics, vol. 79 (2), 377-388. http://dx.doi.org/10.1007/s11192-009-0425-z

Thomson Reuters. (2008). Thomson Scientific Begins Expansion of Web of Science. http://science.thomsonreuters.com/press/2008/8445762/ [consulta: 02/07/2010].

Torres-Salinas, D.; Delgado-López-Cózar, E., and Jiménez-Contreras, E. (2009a). Análisis de la producción de la Universidad de Navarra en revistas de Ciencias Sociales y Humanidades empleando rankings de revistas españolas y la Web of Science. Revista Española de Documentación Científica, vol. 32 (1), 22-39.

Torres-Salinas, D.; Delgado-López-Cózar, E.,and Jiménez-Contreras, E. (2009b). Rankings for departments and researchers within a university using two different databases: Web of Science versus Scopus. Scientometrics, vol. 80 (3), 761-774. http://dx.doi.org/10.1007/s11192-008-2113-9

Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, vol. 62 (1), 117-131. van Leeuwen, T. (2005). The application of bibliometric analyses in the evaluation of social science research. Who benefi ts from it, and why it is still feasible. Scientometrics, vol. 66 (1), 133-154.

van Leeuwen, T. (2007). Modelling of bibliometric approaches and importance of output verifi cation in research performance assessment. Research Evaluation, vol. 16 (2), 93- 105. http://dx.doi.org/10.3152/095820207X227529

van Leeuwen, T., and Moed, H. (2005). Characteristics of Journal Impact Factors: The effect of uncitedness and citation distribution on the understanding of journal impact factors. Scientometrics, vol. 63 (2), 357-371. van Raan, A. F. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, vol. 36 (3), 397-420.

van Raan, A. F. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, vol. 62 (1), 133-143. http://dx.doi.org/10.1007/s11192-005-0008-6

van Raan A. F., and van Leeuwen, T. (2002). Assessment of the scientific basis of interdisciplinary, applied research: Application of bibliometric methods in Nutrition and Food Research. Research Policy, vol. 31 (4), 611-632. http://dx.doi.org/10.1016/S0048-7333(01)00129-9


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS