Alternative Metrics for the Evaluation of Scholarly Activities : An Analysis of Articles Authored by Greek Researchers

Togia, Aspasia and Koseoglou, Eleftheria and Zapounidou, Sofia Alternative Metrics for the Evaluation of Scholarly Activities : An Analysis of Articles Authored by Greek Researchers., 2017 . In 21st International Conference on Electronic Publishing, Limassol, Cyprus, 6-8 June 2017. [Conference paper]

[img]
Preview
 Text
elpub2017.pdf
Download (282kB) | Preview

English abstract

Recently, altmetrics have emerged as alternative means of measuring scholarly impact, aiming at improving and complementing both traditional and web-based metrics. The aim of the present study is to contribute to the altmetrics literature by providing an overview of the coverage of altmetrics sources for the Aristotle University of Thessaloniki (AUTh) publications. We used Scopus to collect all research articles stating AUTh as the affiliation of at least one author and published from 2010 to 2016. The altmetric data originated from Altmetric Explorer, a service provided by Altmetric.com. Only 17% of all publications retrieved from Scopus had some kind of mentions, while there was a clear increasing trend over the years. The presence of altmetrics was different from each Altmetric.com attention source. Around 81% of all mentions came from Twitter. Facebook was a distant second, followed by news outlets. All other sources had very low or negligible coverage. The overwhelming majority of tweets had been posted by members of the public, who do not link to scholarly literature. Medical Sciences had by far the highest number of publications with altmetric scores, followed, in a distance by Sciences. However, Arts, Humanities and Social Sciences publications exhibited a significant altmetric activity. More research is needed in order to get a better insight into the altmetric landscape in Greece and develop an understanding about the kind of influence altmetrics measure, and the relationship, if any, between altmetric indicators and scientific impact.

Item type: Conference paper
Keywords: Bibliometrics, altmetrics, social media metrics, research impact, Greece
Subjects: B. Information use and sociology of information > BB. Bibliometric methods
Depositing user: Sofia Zapounidou
Date deposited: 28 Dec 2017 16:14
Last modified: 28 Dec 2017 16:14
URI: http://hdl.handle.net/10760/32109

References

[1] Togia A, Tsigilis N. Impact factor and education journals: A critical examination and analysis. Int J Educ Res. 2006;45(6):362–79.

[2] Harnad S, Brody T. Comparing the impact of open access (OA) vs. non-OA articles in the same journals. D-Lib Mag [Internet]. 2004 [cited 2016 Nov 30]; 10(6). Available from: http://dlib.org/dlib/june04/harnad/06harnad.html

[3] Radicchi F, Fortunato S, Castellano C. Universality of citation distributions: Toward an objective measure of scientific impact. Proc Natl Acad Sci U S A. 2008; 105(45):17268–72.

[4] Brown LD, Gardner JC. Using citation analysis to assess the impact of journals and articles on Contemporary Accounting Research (CAR). J Account Res. 1985;23(1):84.

[5] Oppenheim C. Using theh-index to rank influential British researchers in information science and librarianship. J Am Soc Inf Sci Technol. 2007;58(2):297–301.

[6] Meho LI, Rogers Y. Citation counting, citation ranking, and h -index of human-computer interaction researchers: A comparison of Scopus and Web of Science. J Am Soc Inf Sci Technol. 2008;59(11):1711–26.

[7] Costas R, Bordons M, van Leeuwen TN, van Raan AFJ. Scaling rules in the science system: Influence of field-specific citation characteristics on the impact of individual researchers. J Am Soc Inf Sci Technol. 2009;60(4):740–53.

[8] Nederhof AJ. Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A review. Scientometrics. 2006;66(1):81–100.

[9] Baumgartner H, Pieters R. The structural influence of marketing journals: A citation analysis of the discipline and its subareas over time. J Mark. 2003;67(2):123–39.

[10] Brody T, Harnad S, Carr L. Earlier Web usage statistics as predictors of later citation impact. J Am Soc Inf Sci Technol. 2006;57(8):1060–72.

[11] Palmer CL, Weber NM, Cragin MH. The analytic potential of scientific data: Understanding re-use value. Proc Am Soc Inf Sci Technol. 2011;48(1):1–10.

[12] MacRoberts MH, MacRoberts BR. Problems of citation analysis. Scientometrics. 1996;36(3):435–44.

[13] Sancho R. Misjudgments and shortcomings in the measurement of scientific activities in less developed countries. Scientometrics. 1992 Jan;23(1):221–33.

[14] MacRoberts M., MacRoberts B. Problems of citation analysis: A critical review. J Am Soc Inf Sci. 1989;40(5):342–9.

[15] Peritz BC. Opinion paper on the objectives of citation analysis: Problems of theory and method. J Am Soc Inf Sci Jul. 1986;43(6).

[16] Aguinis H, Shapiro DL, Antonacopoulou EP, Cummings TG. Scholarly impact: A pluralist conceptualization. Acad Manag Learn Educ. 2014;13(4):623–39.

[17] Priem J, Piwowar HA, Hemminger BM. Altmetrics in the wild: Using social media to explore scholarly impact. arXiv Prepr arXiv12034745 [Internet]. 2012 Mar 20 [cited 2016 Nov 30]; Available from: http://arxiv.org/abs/1203.4745

[18] Björneborn L, Ingwersen P. Toward a basic framework for webometrics. J Am Soc Inf Sci Technol. 2004;55(14):1216–27.

[19] Björneborn L, Ingwersen P. Perspectives of webometrics. Scientometrics. 2001;50(1):65–82.

[20] Galligan F, Dyas-Correia S. Altmetrics: Rethinking the way we measure. Ser Rev. 2013;39(1):56–61.

[21] Howard J. Scholars Seek better ways to track impact online. Chron High Educ [Internet]. 2012; Available from: http://www.chronicle.com/article/As-Scholarship-Goes-Digital/130482/?sid=wc&utm_source=wc&utm_medium=en

[22] altmetrics: a manifesto – altmetrics.org [Internet]. Altmetrics.org. 2017 [cited 30 April 2017]. Available from: http://altmetrics.org/manifesto/

[23] Piwowar H. Introduction altmetrics: What, why and where? Bull Am Soc Inf Sci Technol. 2013;39(4):8–9.

[24] Wouters P, Costas R. Users, Narcissism and control—tracking the impact of scholarly publications in the 21 st century. In: Proceedings of the 17th International Conference on Science and Technology Indicators, Repro-UQAM [Internet]. Montreal, Canada; 2012 [cited 2016 Dec 5]. p. 847–57. Available from: http://sticonference.org/Proceedings/vol2/Wouters_Users_847.pdf

[25] Haustein S, Peters I, Sugimoto CR, Thelwall M, Larivière V. Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. J Assoc Inf Sci Technol. 2014;65(4):656–69.

[26] Sugimoto CR, Work S, Larivière V, Haustein S. Scholarly use of social media and altmetrics: a review of the literature. [Internet]. arXiv preprint arXiv:1608.08112. 2016. Available from: https://arxiv.org/pdf/1608.08112.pdf

[27] Haustein S, Costas R, Larivière V. Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS One. 2015 Mar 17;10(3):e0120495.

[28] Alperin JP. Geographic variation in social media metrics: an analysis of Latin American journal articles. Aslib J Inf Manag. 2015;67(3):289–304.

[29] Hammarfelt B. Using altmetrics for assessing research impact in the humanities. Scientometrics. 2014;101(2):1419–30.

[30] Zahedi Z, Costas R, Wouters P. How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics. 2014;101(2):1491–513.

[31] Costas R, Zahedi Z, Wouters P. Do ‘altmetrics’ correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. J Assoc Inf Sci Technol. 2015;66(10):2003–19.

[32] Haustein S, Peters I, Sugimoto CR, Thelwall M, Larivière V. Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. J Assoc Inf Sci Technol. 2014;65(4):656–69.

[33] Bar-Ilan J, Haustein S, Peters I, Priem J, Shema H, Terliesner J. Beyond citations: Scholars’ visibility on the social Web. In: Proceedings of the 17th International Conference on Science and Technology Indicators, Montreal, Quebec [Internet]. 2012 [cited 2016 Dec 4]. Available from: http://arxiv.org/abs/1205.5611

[34] Li X, Thelwall M, Giustini D. Validating online reference managers for scholarly impact measurement. Scientometrics. 2012;91(2):461–71.

[35] Bar-Ilan J. JASIST 2001-2010. Bull Am Soc Inf Sci Technol. 2012;38(6):24–8.

[36] Costas R, Zahedi Z, Wouters P. The thematic orientation of publications mentioned on social media: Large-scale disciplinary comparison of social media metrics with citations. Aslib J Inf Manag. 2015;67(3):260–88.

[37] Erdt M, Aw AS, Aung HH, Mohammadi E, Theng Y-L. Investigating Singapore’s altmetric landscape. Proc Assoc Inf Sci Technol. 2016;53(1):1–9.

[38] Shuai X, Pepe A, Bollen J, Eysenbach G, Brody T, Harnad S, et al. How the scientific community reacts to newly submitted preprints: Article downloads, twitter mentions, and citations. PLoS One. 2012;7(11):e47523.

[39] Nielsen F. Scientific citations in Wikipedia. First Monday. 2007;12(8).

[40] Groth P, Gurney T. Studying scientific discourse on the web using bibliometrics: A chemistry blogging case study. Web Science Conf [Internet]. 2010 [cited 30 April 2017]. [8 p.]. Available from: http://journal.webscience.org/308/2/websci10_submission_48.pdf

[41] Thelwall M, Haustein S, Larivière V, Sugimoto CR, Seglen P, Wu SNC, et al. Do altmetrics work? Twitter and ten other social web services. PLoS One. 2013;8(5):e64841.

[42] Thelwall M, Wilson P. Mendeley readership altmetrics for medical articles: An analysis of 45 fields. Journal Assoc Inf Sci Technol. 2016;67(8):1962.

[43] Haustein S, Peters I, Bar-Ilan J, Priem J, Shema H, Terliesner J. Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics. 2014;101(2):1145–63.

[44] Haustein S, Larivière V, Thelwall M, Amyot D, Peters I. Tweets vs. Mendeley readers: How do these two social media metrics differ? it - Inf Technol. 2014;56(5):207–15.

[45] Mohammadi E, Thelwall M. Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. J Assoc Inf Sci Technol. 2014;65(8):1627–38.

[46] Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res. 2011;13(4):e123.

[47] Maflahi N, Thelwall M. When are readers as good as citers for bibliometrics? Scopus vs. Mendeley for LIS journals. J Assoc Inf Sci Technol. 2015;67(1):191-199.

[48] Altmetric - Who’s talking about your research? [Internet]. [cited 2017 Mar 31]. Available from: https://www.altmetric.com/

[49] Bornmann L, Haunschild R, Marx W. Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents? Scientometrics.;109(3):1477– 95.

[50] Dinsmore A, Allen L, Dolby K. Alternative perspectives on impact: The potential of ALMs and altmetrics to inform funders about research impact. PLOS Biol. 2014;12(11):e1002003.

[51] Thelwall M, Kousha K. Web indicators for research evaluation. Part 1: Citations and links to academic articles from the Web. El Prof la Inf. 2015;24(5):587.

[52] Samuel GN, Derrick GE. Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the REF2014. Res Eval. 2015;24(3):229–41.

[53] Wilsdon J, Allen L, Belfiore E, Campbell P, Curry S, Hill S. The metric tide: Report of the independent review of the role of metrics in research assessment and management. Bristol: Higher Education Funding Council for England (HEFCE); 2015.

[54] Haunschild R, Bornmann L. How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data. Scientometrics. 2017;110(3):1209–16.

[55] Bik HM, Goldstein MC. An introduction to social media for scientists. PLOS Biol. 2013;11(4):e1001535.

[56] Bosman, J., Kramer, B. 101 innovations in scholarly communication - the changing research workflow [Internet]. figshare. 2017 [cited 29 April 2017]. Available from: https://figshare.com/articles/101_Innovations_in_Scholarly_Communication_the_Changing_Research_Workflow/1286826


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS