Algorithms for Scientific Documents: Past and Present

Álvarez-Llorente, Jesús M., Guerrero-Bote, Vicente P. and De-Moya-Anegón, Félix Algorithms for Scientific Documents: Past and Present. Infonomy, 2025, vol. 3, n. 4. [Journal article (Unpaginated)]

[thumbnail of Research article]
Preview
 Text (Research article)
EN_Alvarez-Guerrero-De-Moya-Algorithms-for-Scientific-Documents.pdf - Published version
Available under License Creative Commons Attribution.
Download (1MB) | Preview

English abstract

This study offers a comprehensive overview of document-level classification algorithms in scientific research, proposed as an alternative to the journal-based categorizations employed by major bibliographic databases such as Web of Science and Scopus. These journal-driven schemes often introduce significant inaccuracies in both information retrieval and research evaluation, as they fail to categorize articles in accordance with their actual content. First, we provide a historical review of the main approaches developed since the emergence of scientific databases, highlighting their contributions as well as their limitations. Automatic clustering techniques and community detection algorithms have represented important advances in the organization of scientific knowledge, yet they cannot serve as a practical substitute for journal-based classifications. Other approaches, such as those relying on neural networks or text mining, face scalability issues that prevent their application at the global level of science. The most recent and promising strategies are built upon simple algorithms that, starting from existing journal categorizations, reclassify articles into the same thematic hierarchies used by bibliographic databases, relying primarily on the analysis of straightforward citation and reference patterns.

Item type: Journal article (Unpaginated)
Keywords: Classification algorithms; Document-level classifications; Classifications; Science classification; Scientific databases; Scientometrics; Citation; Classification schemes; ASJC; Scopus; Web of Science.
Subjects: H. Information sources, supports, channels. > HN. e-journals.
H. Information sources, supports, channels. > HP. e-resources.
I. Information treatment for information services > IC. Index languages, processes and schemes.
Depositing user: Tomàs Baiget
Date deposited: 14 Sep 2025 16:32
Last modified: 14 Sep 2025 16:32
URI: http://hdl.handle.net/10760/47134

References

Althouse, B. M.; West, J. D.; Bergstrom, C.T.; Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the Association for Information Science and Technology, 60(1), 27–34.

https://doi.org/10.1002/asi.20936

Álvarez-Llorente, J. M. (2025). Nuevos algoritmos de clasificación de documentos científicos individuales basados en referencias para mejorar los análisis cienciométricos en las grandes bases de datos de ciencia [Doctoral thesis, University of Extremadura]. Institutional Repository of the University of Extremadura.

Álvarez-Llorente, J. M.; Guerrero‐Bote, V. P.; De-Moya-Anegón, F. (2023). Creating a collection of publications categorized by their research guarantors into the Scopus ASJC scheme. Profesional de la Información, 32(7).

https://doi.org/10.3145/epi.2023.dic.04

Álvarez-Llorente, J. M.; Guerrero-Bote, V. P.; De-Moya-Anegón, F. (2024). New fractional classifications of papers based on two generations of references and on the ASJC Scopus scheme. Scientometrics, 129(6), 3493–3515.

https://doi.org/10.1007/s11192-024-05030-2

Álvarez-Llorente, J. M.; Guerrero-Bote, V. P.; De-Moya-Anegón, F. (2025). New paper-by-paper classification for Scopus based on references reclassified by the origin of the papers citing them. Journal of Informetrics, 19(2), 101647.

https://doi.org/10.1016/j.joi.2025.101647

Andersen, J. P. (2023). Field-level differences in paper and author characteristics across all fields of science in Web of Science, 2000-2020. Quantitative Science Studies, 4(2), 394–422.

https://doi.org/10.1162/qss_a_00246

Blondel, V. D.; Guillaume, J. L.; Lambiotte, R.; Lefebvre, E. (2008). Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment, 2008(10), P10008.

https://doi.org/10.1088/1742-5468/2008/10/P10008

Bornmann, L.; Leydesdorff, L. (2017). Skewness of citation impact data and covariates of citation distributions: A large-scale empirical analysis based on Web of Science data. Journal of Informetrics, 11(1), 164-175.

https://doi.org/10.1016/j.joi.2016.12.001

Bornmann, L.; Tekles, A.; Leydesdorff, L. (2019). How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators. Scientometrics, 119(2), 1187-1205.

http://dx.doi.org/10.1007/s11192-019-03071-6

Boyack, K. W.; Klavans, R. (2010). Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately? Journal of the Association for Information Science and Technology, 61(12), 2389–2404. https://doi.org/10.1002/asi.21419

Boyack, K. W.; Klavans, R. (2020). A comparison of large-scale science models based on textual, direct citation and hybrid relatedness. Quantitative Science Studies (1)4, 1570–1585.

https://doi.org/10.1162/qss_a_00085

Boyack, K. W.; Newman, D.; Duhon, R. J.; Klavans, R.; Patek, M.; Biberstine, J. R.; Schijvenaars, B.; Skupin, A.; Ma, N.; Börner, K. (2011). Clustering more than two million biomedical publications: Comparing the accuracies of nine text-based similarity approaches. PLoS One, 6(3), e18029.

https://doi.org/10.1371/journal.pone.0018029

Boyack, K. W.; Small, H.; Klavans, R. (2013). Improving the Accuracy of Co-citation Clustering Using Full Text. J Am Soc Inf Sci Tec, 64: 1759–1767.

https://doi.org/10.1002/asi.22896

Chumachenko, A.; Kreminskyi, B.; Mosenkis, I.; Yakimenko, A. (2022). Dynamical entropic analysis of scientific concepts. Journal of Information Science, 48(4), 561–569.

https://doi.org/10.1177/0165551520972034

Clauset, A.; Newman, M.; Moore, C. (2004). Finding community structure in very large networks. Physical Review E, 70(6).

https://doi.org/10.1103/physreve.70.066111

De-Moya-Anegón, F.; Herrero-Solana, V.; Jiménez-Contreras, E. (2006). A connectionist and multivariate approach to science maps: the SOM, clustering and MDS applied to library and information science research. Journal of Information Science, 32(1), 63–77.

https://doi.org/10.1177/0165551506059226

Ding, J.; Ahlgren, P.; Yang, L.; Yue, T. (2018). Disciplinary structures in Nature, Science and PNAS: Journal and country levels. Scientometrics, 116(3), 1817–1852. https://link.springer.com/article/10.1007/s11192-018-2812-9

Eykens, J.; Guns, R.; Engels, T. C. E. (2019). Article level classification of publications in sociology: An experimental assessment of supervised machine learning approaches. In: Proceedings of the 17th International Conference on Scientometrics & Informetrics, Rome (Italy), 2–5 September, 738–743.

https://hdl.handle.net/10067/1630240151162165141

Fang, H. (2015). Classifying Research Articles in Multidisciplinary Sciences Journals into Subject Categories. Knowledge Organization, 42(3), 139–153.

https://doi.org/10.5771/0943-7444-2015-3-139

Glänzel, W.; Schubert, A.; Czerwon, H. (1999a). An item-by-item subject classification of papers published in multidisciplinary and general journals using reference analysis. Scientometrics, 44(3), 427–439.

https://doi.org/10.1007/bf02458488

Glänzel, W.; Schubert, A.; Schoepflin, U.; Czerwon, H. (1999b). An item-by-item subject classification of papers published in journals covered by the SSCI database using reference analysis. Scientometrics, 46(3), 431–441.

https://doi.org/10.1007/BF02459602

Glänzel, W.; Thijs, B.; Chi, P. S. (2016). The challenges to expand bibliometric studies from periodical literature to monographic literature with a new data source: the book citation index. Scientometrics, 109, 2165–2179.

https://doi.org/10.1007/s11192-016-2046-7

Glänzel, W.; Thijs, B.; Huang, Y. (2021). Improving the precision of subject assignment for disparity measurement in studies of interdisciplinary research. In: W. Glänzel, S. Heeffer, P. S. Chi, R. Rousseau, Proceedings of the 18th International Conference of the International Society of Scientometrics and Informetrics (ISSI 2021), Leuven University Press, 453–464.

https://kuleuven.limo.libis.be/discovery/fulldisplay?docid=lirias3394551&context=SearchWebhook&vid=32KUL_KUL:Lirias&search_scope=lirias_profile&tab=LIRIAS&adaptor=SearchWebhook&lang=en

Gläser, J.; Glänzel, W.; Scharnhorst, A. (2017). Same data—Different results? Towards a comparative approach to the identification of thematic structures in science. Scientometrics, 111(2), 981–998.

https://doi.org/10.1007/s11192-017-2296-z

Glenisson, P.; Glänzel, W.; Janssens, F.; De-Moor, B. (2005). Combining full text and bibliometric information in mapping scientific disciplines. Information Processing & Management, 41(6), 1548–1572.

https://doi.org/10.1016/j.ipm.2005.03.021

Gómez-Crisóstomo, M. R. (2011). Study and comparison of the Web of Science and Scopus (1996-2007) [Doctoral thesis, University of Extremadura]. Institutional Repository of the University of Extremadura.

Guerrero-Bote, V. P.; De-Moya-Anegón, F. (2012). A further step forward in measuring journals’ scientific prestige: The SJR2 indicator. Journal of informetrics, 6(4), 674-688.

https://doi.org/10.1016/j.joi.2012.07.001

Guerrero-Bote, V.P.; De-Moya-Anegón, F.; Herrero-Solana, V. (2002). Document organization using Kohonen’s algorithm. Information Processing and Management, 38(1), pp. 79-89.

https://doi.org/10.1016/S0306-4573(00)00066-2

Guerrero-Bote, V. P.; Zapico-Alonso, F.; Espinosa-Calvo, M. E.; Gómez-Crisóstomo, R.; De-Moya-Anegón, F. (2007). Import-export of knowledge between scientific subject categories: The iceberg hypothesis. Scientometrics, 71(3), 423–441.

https://doi.org/10.1007/s11192-007-1682-3

Hassan-Montero, Y.; De-Moya-Anegón, F.; Guerrero-Bote, V. P. (2022). SCImago Graphica: a new tool for exploring and visually communicating data. Profesional de la información, 31(5), e310502. https://doi.org/10.3145/epi.2022.sep.02

Hassan-Montero, Y.; Guerrero-Bote, V. P.; De-Moya-Anegón, F. (2014). Graphical interface of the SCImago Journal and Country Rank: an interactive approach to accessing bibliometric information. El profesional de la información, 23(3). https://doi.org/10.3145/epi.2014.may.07

Huang, Y.; Glänzel, W.; Thijs, B.; Porter, A. L.; Zhang, L. (2021). The comparison of various similarity measurement approaches on interdisciplinary indicators (pp. 1–24). FEB - KU Leuven

Janssens, F.; Glänzel, W.; De-Moor, B. (2008). A hybrid mapping of information science. Scientometrics, 75(3), 607–631.

https://doi.org/10.1007/s11192-007-2002-7

Janssens, F.; Leta, J.; Glänzel, W.; De-Moor, B. (2006). Towards mapping library and information science. Information Processing & Management, 42(6), 1614–1642. https://doi.org/10.1016/j.ipm.2006.03.025

Janssens, F.; Zhang, L.; De-Moor, B.; Glänzel, W. (2009). Hybrid clustering for validation and improvement of subject-classification schemes. Information Processing & Management, 45(6), 683–702.

https://doi.org/10.1016/j.ipm.2009.06.003

Javitz, H.; Grimes, T.; Hill, D.; Rapoport, A.; Bell, R.; Fecso, R.; Lehming, R. (2010). U.S. Academic Scientific Publishing. Working paper SRS 11-201. Arlington, VA: National Science Foundation, Division of Science Resources Statistics.

Kandimalla, B.; Rohatgi, S.; Wu, J.; Giles, C. L. (2021). Large scale subject category classification of scholarly papers with deep attentive neural networks. Frontiers in Research Metrics and Analytics, 5, 600382.

https://doi.org/10.3389/frma.2020.600382

Klavans, R.; Boyack, K. W. (2005). Identifying a better measure of relatedness for mapping science. Journal of the Association for Information Science and Technology, 57(2), 251-263.

https://doi.org/10.1002/asi.20274

Klavans, R.; Boyack, K. W. (2006). Quantitative evaluation of large maps of science. Scientometrics, 68, 475–499.

https://doi.org/10.1007/s11192-006-0125-x

Klavans, R.; Boyack, K. W. (2016). Which Type of Citation Analysis Generates the Most Accurate Taxonomy of Scientific and Technical Knowledge? Journal of the Association for Information Science and Technology, 68(4), 984–998.

https://doi.org/10.1002/asi.23734

Lai, K.; Wu, S. (2005). Using the patent co-citation approach to establish a new patent classification system. Information Processing & Management, 41(2), 313–330.

https://doi.org/10.1016/j.ipm.2003.11.004

Lancho-Barrantes, B. S.; Guerrero-Bote, V. P.; De-Moya-Anegón, F. (2010a). The iceberg hypothesis revisited. Scientometrics, 85(2), 443–461.

https://doi.org/10.1007/s11192-010-0209-5

Lancho-Barrantes, B. S.; Guerrero-Bote, V. P.; De-Moya Anegón, F. (2010b). What lies behind the averages and significance of citation indicators in different disciplines? Journal of Information Science, 36(3), 371-382.

https://doi.org/10.1177/0165551510366077

Leydesdorff, L.; De-Moya‐Anegón, F.; Guerrero‐Bote, V. P. (2010). Journal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI. Journal of the American Society for Information Science and Technology, 61(2), 352-369.

https://doi.org/10.1002/asi.21250

Leydesdorff, L.; De-Moya‐Anegón, F.; Guerrero‐Bote, V. P. (2015). Journal maps, interactive overlays, and the measurement of interdisciplinarity on the basis of scopus data (1996–2012). Journal of the Association for Information Science and Technology, 66(5), 1001-1016.

https://doi.org/10.1002/asi.23243

Li, K.; Chen, P.-Y.; Fang, Z. (2019). Disciplinarity of software papers: A preliminary analysis. Proceedings of the Association for Information Science and Technology (56), 706–708.

https://doi.org/10.1002/pra2.143

Marshakova-Shaikevich, I. (2005). Bibliometric maps of field of science. Information Processing & Management, 41(6), 1534–1547.

https://doi.org/10.1016/j.ipm.2005.03.027

McGillivray, B.; Astell, M. (2019). The relationship between usage and citations in an open access mega-journal. Scientometrics, 121, 817–838.

https://doi.org/10.1007/s11192-019-03228-3

Milojević, S. (2020). Practical method to reclassify Web of Science articles into unique subject categories and broad disciplines. Quantitative science studies, 1(1), 183-206.

https://doi.org/10.1162/qss_a_00014

Opthof, T.; Leydesdorff, L. (2010). Caveats for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance. Journal of informetrics, 4(3), 423-430.

https://doi.org/10.1016/j.joi.2010.02.003

Peña-Rocha, M.; Gómez-Crisóstomo, R.; Guerrero-Bote, V. P.; De-Moya-Anegón, F. (2025). Bibliometrics effects of a new paper level classification. Frontiers in Research Metrics and Analytics, 10.

https://doi.org/10.3389/frma.2025.1531758

Rees-Potter, L. K. (1989). Dynamic thesaural systems: A bibliometric study of terminological and conceptual change in sociology and economics with application to the design of dynamic thesaural systems. Information Processing & Management, 25(6), 677–689.

https://doi.org/10.1016/0306-4573(89)90101-5

Sachini, E.; Sioumalas-Christodoulou, K.; Christopoulos, S.; Karampekios, N. (2022) AI for AI: Using AI methods for classifying AI science documents. Quantitative Science Studies, 3(4), 1119–1132.

https://doi.org/10.1162/qss_a_00223

Schildt, H.; Mattsson, J. (2006). A dense network sub-grouping algorithm for co-citation analysis and its implementation in the software tool Sitkis. Scientometrics, 67, 143–163.

https://doi.org/10.1007/s11192-006-0054-8

Shu, F.; Julien, C.; Zhang, L.; Qiu, J.; Zhang, J.; Larivière, V. (2019). Comparing journal and paper level classifications of science. Journal of Informetrics, 13(1), 202–225. https://doi.org/10.1016/j.joi.2018.12.005

Šubelj, L.; Van Eck, N. J.; Waltman, L. (2016). Clustering Scientific Publications Based on Citation Relations: A Systematic Comparison of Different Methods. PLoS one, 11(4), e0154404.

https://doi.org/10.1371/journal.pone.0154404

Thelwall, M.; Pinfield, S. (2024). The accuracy of field classifications for journals in Scopus. Scientometrics, 129(2), 1097–1117.

https://doi.org/10.1007/s11192-023-04901-4

Thijs, B.; Huang, Y.; Glänzel, W. (2021). Comparing different implementations of similarity for disparity and variety measures in studies on interdisciplinarity. FEB Research Report MSI_2103, Report No. MSI_2103.

https://lirias.kuleuven.be/retrieve/610314

Van Eck, N.J.; Waltman, L. (2010). Software survey: VOSviewer, acomputer program for bibliometric mapping. Scientometrics, 84(2), 523–538.

https://doi.org/10.1007/s11192-009-0146-3

Waltman, L.; Boyack, K. W.; Colavizza, G.; Van Eck, N. J. (2020). A principled methodology for comparing relatedness measures for clustering publications. Quantitative Science Studies, 1(2), 691-713.

https://doi.org/10.1162/qss_a_00035

Waltman, L.; Van Eck, N. J. (2012). A new methodology for constructing a publication-level classification system of science. Journal of the Association for Information Science and Technology, 63(12), 2378–2392.

https://doi.org/10.1002/asi.22748

Wang, Q.; Waltman, L. (2016). Large-scale analysis of the accuracy of the journal classification systems of Web of Science and Scopus. Journal of Informetrics, 10(2), 347-364.

https://doi.org/10.1016/j.joi.2016.02.003

Zhang, J.; Shen Z. (2024). Analyzing journal category assignment using a paper-level classification system: multidisciplinary sciences journals. Scientometrics, 129, pp. 5963-5978.

https://doi.org/10.1007/s11192-023-04913-0

Zhang, L.; Janssens, F.; Liang, L.; Glänzel W. (2010). Journal cross-citation analysis for validation and improvement of journal-based subject classification in bibliometric research. Scientometrics, 82, 687–706.

https://doi.org/10.1007/s11192-010-0180-1

Zhang, L.; Rousseau, R.; Glänzel, W. (2016). Diversity of references as an indicator of the interdisciplinarity of journals: Taking similarity between subject fields into account. Journal of the Association for Information Science and Technology, 67(5), 1257-1265.

https://doi.org/10.1002/asi.23487

Zhang, L.; Sun, B.; Shu, F.; Huang, Y. (2022). Comparing paper level classifications across different methods and systems: an investigation of Nature publications. Scientometrics, 127(12), 7633–7651.

https://doi.org/10.1007/s11192-022-04352-3


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS