Citation metrics and strategic mutations of scientific research: narratives and evidence

Guerra, Luca Citation metrics and strategic mutations of scientific research: narratives and evidence. JLIS.it, 2024, vol. 15, n. 1, pp. 144-158. [Journal article (Paginated)]

[thumbnail of jlis_538_1.pdf]
Preview
Text
jlis_538_1.pdf - Published version
Available under License Creative Commons Attribution.

Download (193kB) | Preview

English abstract

Following the diffusion of the management model promoted by the New Public Management also within universities, sectors until then endowed with particular protection, such as the academic one, were progressively subjected to new controls and constraints, which gradually took the form of quantitative surveys, with a growing role of citation metrics. These evaluation processes have given rise to various important critical positions at an international level. If it is true that, according to Goodhart's law, "when a measure becomes an objective it ceases to be a good measure”, it is even more significant to note that when a measure becomes an objective, what is measured ceases to be what it was before. The quantitative measurement of academic performance has in fact triggered forms of gaming such as to alter the very game of scientific research, its purposes and the forms of its sharing. In the article we will carry out a comparison between the forms of mutation and the narratives that accompany them to see to what extent we are legitimate today to talk about the transformation of scientific research into strategic scientific research.

Italian abstract

A seguito della diffusione anche all’interno delle università del modello gestionale promosso dal New public management, settori fino ad allora dotati di una particolare protezione, come quello accademico, si sono progressivamente visti sottoposti a nuovi controlli e vincoli, che hanno progressivamente assunto la forma di rilevazioni quantitative, con un ruolo crescente delle metriche citazionali. Questi processi di valutazione hanno suscitato diverse e importanti prese di posizione critiche sul piano internazionale. Se è vero che, secondo la “legge” di Goodhart “quando una misura diviene un obiettivo smette di essere una buona misura”, ancora più significativo è rilevare che quando una misura diviene un obiettivo, il misurato smette di essere quello che era prima. La misurazione quantitativa delle performance accademiche ha innescato infatti forme di gaming tali da alterare il gioco stesso della ricerca scientifica, dei suoi scopi e delle forme della sua condivisione. Nell’articolo svolgeremo un confronto tra le forme della mutazione e le narrative che le accompagnano per vedere fino a quale punto siamo legittimati oggi a parlare della trasformazione della ricerca scientifica in ricerca scientifica strategica.

Item type: Journal article (Paginated)
Keywords: Scientific misconducts; Gaming metrics; Scholarly communication; Research assesment; Bibliometrics; Mutazioni della ricerca scientifica; Metriche citazionali; Comunicazione scientifica; Valutazione della ricerca; Bibliometria
Date deposited: 15 Sep 2026 05:10
Last modified: 20 Sep 2026 22:24
URI: http://hdl.handle.net/10760/48930

References

Biagioli, Mario. 2016. “Watch out for cheats in citation game”. Nature 535: 201. https://doi.org/10.1038/535201a.

Biagioli, Mario. 2020. “Introduction”. In Gaming the metrics: misconduct and manipulation in academic research, edited by Mario Biagioli and Alexandra Lippman, 1-24. Cambridge, The Mit press. https://doi.org/10.7551/mitpress/11087.003.0006.

Biagioli, Mario and Alexandra Lippman, ed. 2020. Gaming the metrics: misconduct and manipulation in academic research. Cambridge: The Mit press. https://mitpress.mit.edu/9780262537933/gaming-the-metrics.

Brezis, Elise S., and Aliaksandr Birukou. 2020. “Arbitrariness in the Peer Review Process”. Scientometrics, 2123: 393–411. https://doi.org/10.1007/s11192-020-03348-1.

Cari-Ann, Hayer, Mark A. Kaemingk, Jason J. Breeggemann, Daniel Dembkowski, and David Deslauriers. 2013. “Pressures to publish: catalysts for the loss of scientific writing integrity?”. Papers in Natural Resources 689. https://doi.org/10.1080/03632415.2013.813845. https://digitalcommons.unl.edu/natrespapers/689.

Council of the European Union. 2022. “Council conclusions on research assessment and implementations of Open science. Annex 10126/22.” https://data.consilium.europa.eu/doc/document/ST-9515-2022-INIT/en/pdf.

De Bellis, Nicola. 2014. Introduzione alla bibliometria. Roma: AIB.

Dondio, Pierpaolo, Niccolò Casnici, Francisco Grimaldo, Nigel Gilbert, and Flaminio Squazzoni. 2019. “The ‘Invisible hand’ of peer review: the implications of author-referee networks on peer review in a scholarly journal”. Journal of Informetrics 13: 708–716. https://doi.org/10.1016/J..JOI.2019.03.018.

Edwards, A. Mark, and Roy Siddhartha. 2017. “Science is broken. Perverse incentives and the misuse of quantitative metrics have undermined the integrity of scientific research“. Aeon. https://aeon.co/essays/science-is-a-public-good-in-peril-heres-how-to-fix-it.

Fanelli, Daniele. 2020. “Pressures to publish: what effects do we see? In Gaming the metrics: misconduct and manipulation in academic research, edited by Mario Biagioli and Alexandra Lippman, 111-122. Cambridge: The Mit press. https://doi.org/10.7551/mitpress/11087.003.0011.

Fanelli, Daniele, Matteo Schleicher, Ferric C. Fang, Arturo Casadevall, and Elisabeth M. Bik. 2022. “Do individual and institutional predictors of misconduct vary by country? Results of a matched-control analysis of problematic image duplications”. PLoS One 17 (3) e0255334. https://doi.org/10.1371/journal.pone.0255334.

Fang C., Ferric, and Arturo Casadevall. 2011. “Retracted science and the retraction index”. Infective Immunology 79 (10): 3855-9. https://doi.org/10.1128/IAI.05661-11.

Figà-Talamanca, Alessandro. 2000. “L’impact factor nella valutazione della ricerca” [Bozza dell’intervento, Bologna 27 giugno 2000]. http://siba.unipv.it/biblioteche/info/SISSCOWEB_A_Fig_Talamanca_L%27impact_factor_nella_valutazione_della_ricerca_SISSCOWEB.htm.

Garfield, Eugene. 1998. “The impact factor and using it correctly”. De Unfallchirurg, 48 (2): 424. http://garfield.library.upenn.edu/papers/derunfallchirurg_v101%286%29p413y1998english.html.

Gaudino Mario, Brice N. Robinson N., Katia Audisio, Mohamed Rahouma, Umberto Benedetto, Paul Kurlansky, and Stephen E. Fremes. 2021. “Trends and characteristics of retracted articles in the biomedical literature, 1971 to 2020”. JAMA international medicine 181 (8): 1118-1121. https://doi.org/10.1001/jamainternmed.2021.1807.

Hicks Diana, Wouters Paul, Waltman Ludo, De Rijcke Sarah, Rafols Ismael. 2015. “Bibliometrics: The Leiden Manifesto for research metrics”. Nature. 520: 429-431.

Lacetera, Nicola, and Lorenzo Zirulia. 2011. “The economics of scientific misconduct”. The Journal of Law, Economics, and Organization. 27: 568–603. https://doi.org/10.1093/jleo/ewp031.

Martinson, C. Brian, Melissa S. Anderson, and Raymond de Vries. 2005. “Scientists behaving badly”. Nature. 435: 737-738.

Merton, K. Robert. 1998. “The Matthew effect in science, II: cumulative advantage and the symbolism of intellectual property”. Isis. 79, 4: 606-623.

Moher, David, Lex Bouter, Sabine Kleinert, Paul Glasziou, Sham Mai Har, Virginia Barbour, Anne-Marie Coriat, Nicole Foeger, and Ulrich Dirnagl. 2020. “The Hong Kong Principles for assessing researchers: fostering research integrity”. PLoS Biology 18 (7). https://doi.org/10.1371/

journal.pbio.3000737.

Muller, Z. Jerry. 218. The tyranny of metrics. Princeton: Princeton university press.

Naujokaitytė, Goda. 2021. “Number of scientists worldwide reaches 8.8M, as global research spending grows faster than the economy” in Science business. https://sciencebusiness.net/news/number-scientists-worldwide-reaches-88m-global-research-spending-grows-faster-economy.

Oransky, Ivan. 2022. “Retractions are increasing, but not enough”. Nature 608: 9. https://doi.org/10.1038/d41586-022-02071-6.

Pantziarka, Pan, and Lydie Meheus. 2019. “Journal retractions in oncology: a bibliometric study”. Future oncology 15 (31): 3597-3608. https://doi.org/10.2217/fon-2019-0233.

Power, Michael. 1997. The Audit Society. Oxford: Oxford University Press.

Smith, Richard. 2006. “Peer review: a flawed process at the heart of science and Journals”. Journal of the Royal society of medicine 99 (4): 178-182.

Tijdink, K. Joeri, Anton C.M. Vergouwen, and Yvo M. Smulders. 2013. “Publication pressure and burn out among Dutch medical professors: a nationwide survey”. PLoS One 8 (9): e7338. https://doi.org/10.1371/journal.pone.0073381.

Turbanti, Simona. 2018. Strumenti di misurazione della ricerca. Milano: Editrice Bibliografica.

Wilsdon, James, ed. 2015. The metric tide: report of the independent review of the role of metrics in research assessment and management. Thousand Oaks: SAGE publications. https://doi.org/10.13140/RG.2.1.4929.1363. https://sk.sagepub.com/books/the-metric-tide.

Wouters, Paul. 2020. “The mismeasurement of quality and impact”. In Gaming the metrics: misconduct and manipulation in academic research, edited by Mario Biagioli and Alexandra Lippman, 66-75. Cambridge, The Mit press. https://doi.org/10.7551/mitpress/11087.003.0006.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item