Peer review in research assessment and data analysis of Italian publications in SSD M-STO/08 (Archival science, bibliography, library science)

Morriello, Rossana Peer review in research assessment and data analysis of Italian publications in SSD M-STO/08 (Archival science, bibliography, library science). JLIS.it, 2023, vol. 14, n. 1, pp. 99-120. [Journal article (Paginated)]

[thumbnail of jlis_510_1.pdf]
Preview
 Text
jlis_510_1.pdf - Published version
Available under License Creative Commons Attribution.
Download (1MB) | Preview

English abstract

Since the introduction of research assessment systems at institutional level in the 1980s, the ongoing debate on the roles and functions of peer review and bibliometrics has been vivid and lively. In the first part of the article, the main lines over time of this debate are traced, and a reflection on the epistemic functions of peer review and citations is proposed. In Italy, the first research assessment exercise (VTR) was based on peer review only, while the following ones (VQR) were based on different methods for bibliometric disciplines and non-bibliometric disciplines, namely bibliometric indicators and peer review. Starting from a data analysis on Italian publications, and using as a sample data from M-STO/08 (Archival science, bibliography and library science) area, the essay shows some trends and changes in publication habits in HSS. Conclusions open a perspective on revitalization of peer review as a solid qualitative method for research assessment.

Italian abstract

Fin dall’introduzione dei sistemi di valutazione della ricerca a livello istituzionale negli anni Ottanta del XX secolo, è stato vivo e acceso il dibattito sul ruolo della peer review rispetto agli indicatori bibliometrici, di cui nella prima parte del saggio si riportano le linee principali, affiancate da una riflessione sulle funzioni epistemiche della peer review e della citazione. In Italia, il primo esercizio di Valutazione Triennale della Ricerca (VTR) era basato esclusivamente sulla peer review mentre nei successivi esercizi di Valutazione della Qualità della Ricerca (VQR) si è consolidata una distinzione tra gli indicatori per i settori bibliometrici e la peer review per i settori non bibliometrici. Partendo dai dati dall’analisi dei dati sulle pubblicazioni italiane negli anni della VQR, esemplificate tramite l’SSD M-STO/08 (Archivistica, bibliografia e biblioteeconomia), il saggio si sofferma poi su alcune tendenze e cambiamenti rilevabili nelle abitudini di pubblicazione dei settori HSS. Le conclusioni aprono una prospettiva di rivitalizzazione della peer review come solido strumento qualitativo per la valutazione della ricerca.

Item type: Journal article (Paginated)
Keywords: Peer review; Bibliometrics; Research assessment; VQR; Academic field M-STO/08 (Archival science, bibliography and library science); HSS; Humanities and social sciences; Academic research; Bibliometria; Valutazione della ricerca; Settore scientifico M-STO/08 (Archivistica, bibliografia e biblioteconomia); Scienze sociali e umanistiche; Ricerca accademica
Date deposited: 15 Sep 2026 05:10
Last modified: 20 Sep 2026 22:24
URI: http://hdl.handle.net/10760/48913

References

Abramo, Giovanni. 2017. “Correspondence On tit for tat: Franceschini and Maisano versus ANVUR regarding the Italian research assessment exercise VQR 2011–2014”. Journal of Infometrics 11: 783-787.

Abramo, Giovanni, Ciriaco Andrea D’Angelo, e Alessandro Caprasecca. 2009. “Allocative efficiency in public research funding: Can bibliometrics help?”. Research Policy 38: 206-215.

ANVUR (Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca). 2013. Rapporto finale sulla VQR 2004-2010. https://www.anvur.it/rapporto/.

ANVUR (Agenzia Nazionale di Valutazione del Sistema Universitario e della Ricerca). 2018. Rapporto biennale sullo stato del sistema universitario e della ricerca. https://www.anvur.it/documenti-ufficiali/rapporti-sullo-stato/.

Baccini, Alberto, e Giuseppe De Nicolao. 2016. “Do they agree? Bibliometric evaluation versus informed peer review in the Italian research assessment exercise”. Scientometrics 108: 1651–1671.

Baldwin, Melinda. 2020. “Peer Review,” in Encyclopedia of the History of Science. Carnegie Mellon University. https://lps.library.cmu.edu/ETHOS/article/id/38/.

Bence, Valerie e Charles Oppenheim. 2005. “The Evolution of the UK’s Research Assessment Exercise: Publications, Performance and Perceptions”. Journal of Educational Administration and History. 37 (2): 137-155. https://doi.org/10.1080/00220620500211189.

Benedetto, Sergio, Daniele Checchi, Andrea Graziosi, e Marco Malgarini. 2017. “Comments on the paper Critical remarks on the Italian assessment exercise”. Journal of Informetrics, 11 (2): 622–624.

Bertocchi, Graziella, Alfonso Gambardella, Tullio Jappelli, Carmela A. Nappi, e Franco Peracchi. 2015. “Bibliometric evaluation vs. informed peer review: Evidence from Italy”. Research Policy 44 (2): 451-466.

Bulaitis, Zoe Hope. 2021. ““Minimum expectations” are no way to value the arts, humanities, and social sciences”. LSE Blog, June 7th. https://blogs.lse.ac.uk/impactofsocialsciences/2021/06/07/minimum-expectations-are-no-way-to-value-the-arts-humanities-and-social-sciences/.

Bush, Vannevar. 2013. Manifesto per la rinascita di una nazione. Scienza, la frontiera infinita. Torino: Bollati Boringhieri.

Castellucci, Paola. 2020. “La visione del giudizio. Una prospettiva romantica,” in Libri, biblioteche e società. Studi per Rosa Marisa Borraccini, a cura di Alberto Petrucciani, Valentina Sestini, Federico Valacchi. Macerata: EUM.

Cronin, Blaise. 2005. The Hand of Science. Academic Writing and Its Rewards. Lanham: The Scarecrow Press.

Demetrescu, Camil, Andrea Ribichini, e Marco Schaerf. 2020. “Are Italian research assessment exercises size‑biased?”. Scientometrics 125: 533-549.

EUA (European University Association), Science Europe, Karen Stroobants, e (EC) European Commission. 2022. Agreement on Reforming Research Assessment. https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/reforming-research-assessment-agreement-now-final-2022-07-20_en.

Faggiolani, Chiara. 2015. La bibliometria. Roma: Carocci.

Franceschet, Massimo, e Antonio Costantini. “The first Italian research assessment exercise: A bibliometric perspective”. Journal of Infometrics 5: 275-291.

Franceschini, Fiorenzo, e Domenico Maisano. 2017. “Critical remarks on the Italian research assessment exercise VQR 2011–2014”. Journal of Infometrics 11: 337-357.

Galderisi, Claudio, Mauro Perretti, Nuria Sebastian Galles, e Thed van Leeuwen. 2019. Report of the Group of Experts charged by ANVUR to advice on the process ‘Valutazione della Qualità della Ricerca (VQR)’. An independent assessment on the past VQRs carried out by ANVUR. http://www.anvur.it/wp-content/uploads/2019/03/High-Experts-Report-on-VQR.pdf.

Garfield, Eugene. 1973. “Citation Index depend upon the paper not the journal! Don’t count on ‘citation by association’”, Current Contents 22: 5-6. http://www.garfield.library.upenn.edu/essays/V1p452y1962-73.pdf.

Garfield, Eugene. 1996. “How can impact factors be improved?” British Medical Journal, 313: 411–413, https://doi.org/10.1136/bmj.313.7054.411

Geuna, Aldo, e Matteo Piolatto. 2016. “Research assessment in the UK and Italy: Costly and difficult, but probably worth it (at least for a while)”. Research Policy 45 (1): 260-271.

Gingras, Yves. 2016. Bibliometrics and Research Evaluation: Uses and Abuses. Cambridge (Ma): MIT Press.

Greco, Pietro. 2013. “Introduzione,” in V. Bush. Manifesto per la rinascita di una nazione. Scienza, la frontiera infinita. Torino: Bollati Boringhieri.

Greco, Pietro. 2019. La scienza e l’Europa. Dal secondo dopoguerra a oggi. Roma: L’Asino D’Oro.

Greenfield, Nathan M. 2022. “The unkindest cut? – Behind the paring of the humanities”. University World News. 13 August 2022. https://www.universityworldnews.com/post.php?story=20220808152834589.

Guerrini, Mauro. 2022. “Sua Maestà il revisore: alcune considerazioni sul processo di peer-review all'interno della LIS”. AIB Studi, 61(3): 585–592. https://doi.org/10.2426/aibstudi-13328.

Kaplan, Norman. 1965. “The Norms of Citation Behavior: Prolegomena to the Footnote”. American Documentation 16 (3): 179-184.

Meltzer, Leo. 1956. “Scientific Productivity in Organizational Settings”. Journal of Social Issues 12: 32-40.

Morriello, Rossana. 2019. “La valutazione della ricerca in Olanda. Un’analisi comparativa con il sistema italiano”. Biblioteche oggi 37 ottobre: 33-41.

Myklebust, Jan Petter. 2021. “Copenhagen University set to cut humanities places by 24%”. University World News. 20 October. https://www.universityworldnews.com/post.php?story=20211020154347298.

Phillimore, A.J. 1989. “University research performance indicators in practice: The University Grants Committee’s evaluation of British universities, 1985-86”. Research Policy 18: 255-171.

Polanyi, Michael. 1962. “The Republic of Science: Its Political and Economic Theory”. Minerva I (1): 1-32. https://www.jstor.org/stable/41821153.

Polanyi, Michael. 2006. La società libera. Pensieri liberali, a cura di Massimo Baldini e Antonello Malavasi. Roma: Armando Editore.

Popper, Karl. 2018. La società aperta e i suoi nemici. Roma: Armando Editore.

Price, Derek de Solla. 1967. Sociologia della creatività scientifica. Milano, Bompiani.

Pritchard, Alan. 1969. “Statistical bibliography or bibliometrics”. Journal of Documentation (25): 348-349.

Rossi, Paolo. 2016. “Stato giuridico, reclutamento ed evoluzione della docenza universitaria (1975-2015)”. RT. A Journal on Research Policy & Evaluation 1. https://doi.org/10.13130/2282-5398/6726.

Solimine, Giovanni. 2016. “Impatto e valutazione della ricerca scientifica”. Nuovi Annali della Scuola speciale per archivisti e bibliotecari. 30: 235-251.

Solimine, Giovanni. 2017. Progetto LI.B.RO. For a Liable Evaluation of Book’s Role in Socio-Economic Sciences and Humanities: an International Comparison. Report finale. ANVUR, Working Paper 2017/06.

Villa, Maria Luisa. 2018. Scienza è democrazia. Come funziona il mondo della ricerca. Postfazione di Pietro Greco. Milano: Guerini e Associati.

VSNU (Association of Cooperating Universities in the Netherlands), KNAW (Royal Netherlands Academy of Arts and Sciences), e NWO (Dutch Research Council). 2020. Strategy Evaluation Protocol (SEP). https://www.universiteitenvannederland.nl/files/documenten/Domeinen/Onderzoek/SEP_2021-2027.pdf.

Weingart, Peter. 2005. “Impact of bibliometrics upon the science system: Inadvertent consequences?”. Scientometrics 62 (1): 117-131.

Wouters, Paul. 1998. “The Signs of Science”. Scientometrics 41 (1-2): 224-241.

Wouters, Paul. 2016. “Semiotics and Citations,” in Theories of Infometrics and Scholarly Communication. A Festschrift in honor of Blaise Cronin, edited by Cassidy R. Sugimoto. Berlin: De Gruyter.

Zuckerman, Harriet, e Robert K. Merton. 1971. “Patterns of Evaluation in Science: Institutionalisation, Structure and Functions of the Referee System”. Minerva 9 (1): 66-100.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS