Bibliometrics versus altmetrics: Researchers’ attitudes in Slovakia

Melicherová, Michaela, Ondrišová, Miriam and Šušol, Jaroslav Bibliometrics versus altmetrics: Researchers’ attitudes in Slovakia. Iberoamerican Journal of Science Measurement and Communication, 2021, vol. 1, n. 1. [Journal article (Unpaginated)]

[thumbnail of Bibliometrics_versus_altmetrics.pdf]
Preview
 Text
Bibliometrics_versus_altmetrics.pdf
Download (522kB) | Preview

English abstract

Objective. The paper presents principal theoretical starting points and an overview of attitudes to altmetrics-based research evaluation among scientists of various disciplines in Slovakia, more specifically in comparison with traditional, bibliometric approaches. Design/Methodology/Approach. We used a questionnaire and asked researchers from various disciplines about their knowledge of principle characteristics defining the altmetrics and about their attitudes towards the application of bibliometrics and altmetrics approaches in the processes of research evaluation. The questionnaire covered a wide range of issues, predominantly associated with interpretation and perception of the importance of alternative indicators, including the factors influencing the interpretation. In this analysis, we concentrate on those parts of research reflecting the willingness of respondents to accept alternative indicators as part of research/science assessment. Results/Discussion. The attitude to the application of alternative indicators in research assessment is not significantly more negative among Slovak scientists if we compare it to other countries. The citations are still a respected indicator perceived as a tool for creating bonds within science. If they are perceived as obsolete or insufficient, it relates to the experience of researchers whose results are predominantly aimed at non-academic target group or if they are active in the research of social phenomena and problems. Due to pragmatic reasons connected with research financing, however, registering the citations is a generally accepted priority. Familiarity and practical experience with social media were identified as one of the factors influencing the interpretation of alternative indicators in the environment of both public and academic media. Conclusions. Traditional bibliometric methods, especially in the area of research evaluation, are still preferred over the altmetrics. The scientists realize that the kind of attention captured by the indicators based on social media is not comparable with the citation-based impact. Or, more precisely, that altmetrics reflects a different kind of impact whose correspondence with a traditional world of bibliographic citations remains to be explored. Originality/Value. The article presents the results of original research carried out in the community of scientists in Slovakia, applying quantitative research methods. It contributes to the results of previous studies on the attitudes and acceptance of altmetrics in the research community, with a special focus on research assessment.

Item type: Journal article (Unpaginated)
Keywords: Altmetric indicators, Bibliometrics, Research evaluation, Researchers’ attitudes
Subjects: B. Information use and sociology of information > BB. Bibliometric methods
Depositing user: Unnamed user with email ijsmc@colnes.org
Date deposited: 15 Feb 2021 23:09
Last modified: 15 Feb 2021 23:09
URI: http://hdl.handle.net/10760/40969

References

Alhoori, H., & Furuta, R. (2014). Do altmetrics follow the crowd or does the crowd follow altmetrics?. In IEEE/ACM Joint Conference on Digital Libraries (pp. 375-378). IEEE. https://dl.acm.org/doi/10.5555/2740769.2740833

Amath, A., Ambacher, K., Leddy, J. J., Wood, T. J., & Ramnanan, C. J. (2017). Comparing alternative and traditional dissemination metrics in medical education. Medical education, 51(9), 935-941. https://doi.org/10.1111/medu.13359

Banshal, S. K., Basu, A., Singh, V. K., & Muhuri, P. K. (2018). Scientific vs. public attention: a comparison of top cited papers in WoS and top papers by Altmetric Score. In International Workshop on Altmetrics for Research Outputs Measurements and Scholarly Information Management (pp. 81-95). Springer, Singapore. https://doi.org/10.1007/978-981-13-1053-9_7

Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of informetrics, 8(4), 895-903. https://doi.org/10.1016/j.joi.2014.09.005

Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123-1144. https://doi.org/10.1007/s11192-015-1565-y

Bornmann, L., & Haunschild, R. (2017). Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact?. Scientometrics, 110(2), 937-943. https://doi.org/10.1007/s11192-016-2200-2

Bucknell, T. (2014). Making sense and making use of Altmetrics in research evaluation. In Septentrio Conference Series (No. 1). https://doi.org/10.7557/5.3230

Büttgen, S. & Luprich, J. (2015). Alternativní metriky: měření dopadu výzkumu právě teď. In Infos (EBSCO). http://www.infolib.sk/files/infos_2015_prezentacie/ebscoinfos-2015-altmetrie.pdf

Copiello, S., & Bonifaci, P. (2018). A few remarks on ResearchGate score and academic reputation. Scientometrics, 114(1), 301-306. https://doi.org/10.1007/s11192-017-2582-9

Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003-2019. https://doi.org/10.1002/asi.23309

Costas, R., Zahedi, Z., & Wouters, P. (2015b). The thematic orientation of publications mentioned on social media. Aslib Journal of Information Management, 67(3), 260. https://doi.org/10.1108/AJIM-12-2014-0173

D’Angelo, C. A., & Di Russo, S. (2019). Testing for universality of Mendeley readership distributions. Journal of informetrics, 13(2), 726-737. https://doi.org/10.1016/j.joi.2019.03.011

Díaz-Faes, A. A., Bowman, T. D., & Costas, R. (2019). Towards a second generation of ‘social media metrics’: Characterizing Twitter communities of attention around science. PloS one, 14(5), e0216408. https://doi.org/10.1371/journal.pone.0216408

Engwall, L., Blockmans, W., & Weaire, D. (2014). Bibliometrics: Issues and Context. In W. Blockmans, L. Engwall & D.Weaire (Eds.), Bibliometrics : Usa and Abuse in the Review of Research Performance (pp. 1–7). London. Retrieved from https://portlandpress.com/pages/vol_87_bibliometrics_use_and_abuse_in_the_review_of_research_performance

Erdt, M., Nagarajan, A., Sin, S. C. J., & Theng, Y. L. (2016). Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media. Scientometrics, 109(2), 1117-1166. https://doi.org/10.1007/s11192-016-2077-0

Gutiérrez-Salcedo, M., Martínez, M. Á., Moral-Muñoz, J. A., Herrera-Viedma, E., & Cobo, M. J. (2018). Some bibliometric procedures for analyzing and evaluating research fields. Applied intelligence, 48(5), 1275-1287. https://doi.org/10.1007/s10489-017-1105-y

Haustein, S., Bowman, T. D., & Costas, R. (2016). Interpreting Altmetrics: Viewing Acts on Social Media through the Lens of Citation and Social Theories. In Sugimoto CR (Ed.), Theories of Informetrics and Scholarly Communication (pp. 372–406). Retrieve from https://pdfs.semanticscholar.org/7cad/8c2212afb03a0d3e61daa630357c9282c040.pdf

Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PloS one, 10(3), e0120495. https://doi.org/10.1371/journal.pone.0120495

Holmberg, K., Bowman, S., Bowman, T., Didegah, F., & Kortelainen, T. (2019). What Is Societal Impact and Where Do Altmetrics Fit into the Equation?. Journal of Altmetrics, 2(1). http://doi.org/10.29024/joa.21

Holmberg, K., & Vainio, J. (2018). Why do some research articles receive more online attention and higher altmetrics? Reasons for online success according to the authors. Scientometrics, 116(1), 435-447. https://doi.org/10.1007/s11192-018-2710-1

Južnič, P., Vilar, P., & Bartol, T. (2014). What do researchers think about altmetrics and are they familiar with their abilities?. Libraries in the Digital Age (LIDA) Proceedings, 13. Retrieved from http://ozk.unizd.hr/proceedings/index.php/lida/article/view/128/131

Maflahi, N., & Thelwall, M. (2016). When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals. Journal of the Association for information Science and Technology, 67(1), 191-199. https://doi.org/10.1002/asi.23369

Mohammadi, E., Thelwall, M., Haustein, S., & Larivière, V. (2015). Who reads research articles? An altmetrics analysis of Mendeley user categories. Journal of the Association for Information Science and Technology, 66(9), 1832-1846. https://doi.org/10.1002/asi.23286

Mohammadi, E., & Thelwall, M. (2014). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the Association for Information Science and Technology, 65(8), 1627-1638. https://doi.org/10.1002/asi.23071

Mohammadi, E., Thelwall, M., Kwasny, M., & Holmes, K. L. (2018). Academic information on Twitter: A user survey. PloS one, 13(5), e0197265. https://doi.org/10.1371/journal.pone.0197265

Motal, J. (2017). Publikační maraton zadupává svobodnou diskusi na univerzitách. Deník Referendum. http://denikreferendum.cz/clanek/26245-publikacni-maraton-zadupava-svobodnou-diskusi-na-univerzitach

Nuzzolese, A. G., Ciancarini, P., Gangemi, A., Peroni, S., Poggi, F., & Presutti, V. (2019). Do altmetrics work for assessing research quality?. Scientometrics, 118(2), 539-562. https://doi.org/10.1007/s11192-018-2988-z

Ondrišová, M. (2016). Alternatívne hodnotenia vplyvu vedeckých výstupov vo webovom prostredí. In Knižničná a informačná veda, 26, 28-44. Retrieved from https://fphil.uniba.sk/fileadmin/fif/katedry_pracoviska/kkiv/Rozne/Zbornik_Kniznicna_a_informacna_veda_26_zbornikkkivxxvi_01.pdf

Ringelhan, S., Wollersheim, J., & Welpe, I. M. (2015). I like, I cite? Do Facebook likes predict the impact of scientific work?. PloS one, 10(8), e0134389. https://doi.org/10.1371/journal.pone.0134389

Robinson-Garcia, N., van Leeuwen, T. N., & Rafols, I. (2018). Using altmetrics for contextualised mapping of societal impact: From hits to networks. Science and Public Policy, 45(6), 815-826. https://doi.org/10.1093/scipol/scy024

Roemer, R. C., & Borchardt, R. (2015). Meaningful metrics: A 21st-century librarian's guide to bibliometrics, altmetrics, and research impact. American Library Association. Retrieved from http://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/booksanddigitalresources/digital/9780838987568_metrics_OA.pdf

Saenen, B., & Borrell-Damián, L. (2019). Reflections on University Research Assessment: key concepts, issues and actors. European University Association Briefing. Retrieved from https://eua.eu/resources/publications/825:reflections-on-university-research-assessment-key-concepts,-issues-and-actors.html

Setti, G. (2013). Bibliometric indicators: Why do we need more than one?. IEEE Access, 1, 232-246.

Sugimoto, C. (2015). Attention is not impact" and other challenges for altmetrics. Discover the future of research. Retrieved from https://hub.wiley.com/community/exchanges/discover/blog/2015/06/23/attention-is-not-impact-and-other-challenges-for-altmetrics

Sugimoto, C. R., Work, S., Lariviére, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–2062. DOI: https://doi.org/10.1002/asi.23833

Thelwall, M. (2020). Measuring societal impacts of research with altmetrics? Common problems and mistakes. Journal of Economic Surveys. https://doi.org/10.1111/joes.12381

Vanclay, F., Esteves, A. M., Aucamp, I., & Franks, D. M. (2015). Social Impact Assessment: Guidance for assessing and managing the social impacts of projects. Fargo ND: International Association for Impact Assessment, (p. 170). Retrieved from http://espace.library.uq.edu.au/view/UQ:355365/UQ355365.pdf

Wang, M., Wang, Z., & Chen, G. (2019). Which can better predict the future success of articles? Bibliometric indices or alternative metrics. Scientometrics, 119(3), 1575-1595. https://doi.org/10.1007/s11192-019-03052-9

Wang, X., & Inaba, M. (2009). Analyzing structures and evolution of digital humanities based on correspondence analysis and co-word analysis. Art research, 9, 123-134.

Wouters, P., Zahedi, Z., & Costas, R. (2019). Social media metrics for new research evaluation. In Glänzel, W., Moed, H. F., Schmoch, U., & Thelwall, M. (Eds.), Springer handbook of science and technology indicators (pp. 687-713). Springer, Cham. https://doi.org/10.1007/978-3-030-02511-3_26

Xia, F., Su, X., Wang, W., Zhang, C., Ning, Z., & Lee, I. (2016). Bibliographic analysis of Nature based on Twitter and Facebook altmetrics data. PloS one, 11(12), e0165997. https://doi.org/10.1371/journal.pone.0165997

Yu, H. (2017). Context of altmetrics data matters: an investigation of count type and user category. Scientometrics, 111(1), 267-283. https://doi.org/10.1007/s11192-017-2251-z.

Zahedi, Z., Costas, R., Larivière, V., & Haustein, S. (2016). What makes papers visible on social media? An analysis of various document characteristics. In Proceedings of the 21th International Conference on Science and Technology Indicators (pp. 1196-1199). Universitat Politècnica de Valencia. https://arxiv.org/abs/1703.05777

Zhang, X., Wang, X., Zhao, H., de Pablos, P. O., Sun, Y., & Xiong, H. (2019). An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications. Scientometrics, 119(3), 1311-1344. https://doi.org/10.1007/s11192-019-03088-x


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS