Evaluating the impact of semantic technologies on bibliographic systems: A user-centred and comparative approach

Rico, Mariano, Vila-Suero, Daniel, Botezan, Iuliana and Gómez-Pérez, Asunción Evaluating the impact of semantic technologies on bibliographic systems: A user-centred and comparative approach. Journal of Web Semantics, 2019, vol. 59, p. 100500. [Journal article (Paginated)]

[thumbnail of Evaluating-the-impact.pdf]
Preview
 Text
Evaluating-the-impact.pdf
Download (2MB) | Preview

English abstract

Semantic and linked-data technologies are currently used by several cultural heritage institutions to make their content available through the Web. Although these technologies are heavily oriented towards data reuse and integration, one clear benefit highlighted by recent literature is the enhancement of human cultural consumption and user experience through the development of novel cultural end-user applications like Online Public Access Catalogues (OPACs). However, to the best of our knowledge, studies into the impact of these technologies on end-user applications are scarce. In order to address this lack, we report the results of two within-group user-centred studies of two online bibliographic systems in a realistic setting — using a widely deployed OPAC and its counterpart linked-data based system, datos.bne.es. The results of our first within-group study show that users of the system based on linked data required significantly less time and visited fewer pages to complete a typical search and retrieval activity. Additionally, the results of our user satisfaction tests also provided significantly better results for this new system. These results are consistent with the hypothesis that semantic technologies applied to library catalogues provide an enhancement that helps satisfy users’ information needs.

Item type: Journal article (Paginated)
Keywords: User-centred evaluation, Usability, Cultural heritage, Digital humanities, Linked data, Ontologies
Subjects: L. Information technology and library technology
L. Information technology and library technology > LQ. Library automation systems.
Depositing user: Iuliana Botezan
Date deposited: 29 Apr 2021 10:04
Last modified: 01 May 2021 15:01
URI: http://hdl.handle.net/10760/41967

References

[1] T. Heath, C. Bizer, Linked data: Evolving the web into a global data space, Synth. Lect. Semantic Web Theory Technol. 1 (1) (2011) 1–136.

[2] B. Haslhofer, A. Isaac, data.europeana.eu: The europeana linked open data pilot, in: International Conference on Dublin Core and Metadata Applications, 2011, pp. 94–104.

[3] A. Simon, R. Wenz, V. Michel, A. Di Mascio, Publishing bibliographic records on the web of data: opportunities for the BnF (French National Library), in: The Semantic Web: Semantics and Big Data, Springer, 2013, pp. 563–577.

[4] H. Al-Kilidar, K. Cox, B. Kitchenham, The use and usefulness of the ISO/IEC 9126 quality standard, in: Empirical Software Engineering, 2005. 2005 International Symposium on, IEEE, 2005, pp. 126–132.

[5] D. Vila Suero, E. Escolano Rodríguez, Linked data at the spanish national library and the application of IFLA RDFS models, IFLA ScatNews 1 (35) (2011) 5–6.

[6] D. Vila-Suero, B. Villazón-Terrazas, A. Gómez-Pérez, datos.bne.es: A library linked dataset, Semantic Web J. 4 (3) (2013) 307–313.

[7] D. Vila-Suero, A. Gómez-Pérez, datos.bne.es and MARiMbA: an insight into library linked data, Library Hi Tech. 31 (4) (2013) 575–601.

[8] G. Charness, U. Gneezy, M.A. Kuhn, Experimental methods: Betweensubject and within-subject design, J. Econ. Behav. Organ. 81 (1) (2012) 1–8.

[9] P. Ingwersen, K. Järvelin, The Turn: Integration of Information Seeking and Retrieval in Context, Springer, 2005.

[10] G. Cumming, F. Fidler, D.L. Vaux, Error bars in experimental biology, J. Cell Biol. 177 (1) (2007) 7–11.

[11] B. Efron, R. Tibshirani, Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy, Statist. Sci. 1 (1) (1986) 54–75, http://www.jstor.org/stable/2245500.

[12] J.P. Chin, V.A. Diehl, K.L. Norman, Development of an instrument measuring user satisfaction of the human-computer interface, in: E. Soloway, D. Frye, S.B. Sheppard (Eds.), Interface Evaluations. Proceedings of ACM CHI’88 Conference on Human Factors in Computing Systems, 1988, pp. 213–218.

[13] M. Rico, O. Corcho, V. Méndez, J.M. Gómez-Pérez, A semantically enhanced UPnP control point for sharing multimedia content, IEEE Internet Comput. 15 (6) (2011) 58–64.

[14] M. Rico, J. Ramírez, D. Riofrío-Luzcando, M. Berrocal-Lobo, A cost-effective approach for procedural training in virtual worlds, J. UCS 23 (2) (2017) 208–232.

[15] P. Borlund, P. Ingwersen, The development of a method for the evaluation of interactive information retrieval systems, J. Doc. 53 (3) (1997) 225–250.

[16] P. Borlund, Experimental components for the evaluation of interactive information retrieval systems, J. Doc. 56 (1) (2000) 71–90.

[17] P. Borlund, The IIR evaluation model: a framework for evaluation of interactive information retrieval systems, Inform. Res. 8 (3) (2003).

[18] B. Ricardo, R. Berthier, Modern Information Retrieval: The Concepts and Technology Behind Search, second ed., Addision Wesley, 2011.

[19] A. Díaz, A. García, P. Gervás, User-centred versus system-centred evaluation of a personalization system, Inform. Process. Manag. 44 (3) (2008) 1293–1307.

[20] D. Petrelli, On the role of user-centred evaluation in the advancement of interactive information retrieval, Inform. Process. Manag. 44 (1) (2008) 22–38.

[21] A. Spink, A user-centered approach to evaluating human interaction with web search engines: an exploratory study, Inform. Process. Manag. 38 (3) (2002) 401–426.

[22] L. Van Velsen, T. Van Der Geest, R. Klaassen, M. Steehouder, User-centered evaluation of adaptive and adaptable systems: a literature review, Knowl. Eng. Rev. 23 (03) (2008) 261–281.

[23] T. Merčun, M. Žumer, T. Aalberg, Presenting bibliographic families using information visualization: Evaluation of FRBR-based prototype and hierarchical visualizations, J. Assoc. Inform. Sci. Technol. 68 (2) (2017) 392–411.

[24] T. Baker, E. Bermès, K. Coyle, G. Dunsire, A. Isaac, P. Murray, M. Panzer, J. Schneider, R. Singer, E. Summers, W. Waites, J. Young, M. Zeng, Library linked data incubator group: final report, Tech. Rep., World Wide Web Consortium (W3C), 2011.

[25] T. Berners-Lee, J. Hollenbach, K. Lu, J. Presbrey, E. Prud’hommeaux, M.M.C. Schraefel, Tabulator redux: browsing and writing linked data, in: C. Bizer, T. Heath, K. Idehen, T. Berners-Lee (Eds.), LDOW, in: CEUR Workshop Proceedings, vol. 369, CEUR-WS.org, 2008.

[26] C. Christou, C. Angus, C. Loscos, A. Dettori, M. Roussou, A versatile large-scale multimodal vr system for cultural heritage visualization, in: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, ACM, 2006, pp. 133–140.

[27] F. Monaghan, G. Bordea, K. Samp, P. Buitelaar, Exploring your research: sprinkling some saffron on semantic web dog food, in: Semantic Web Challenge At the International Semantic Web Conference, vol. 117, 2010, pp. 420–435.

[28] J. Nandzik, B. Litz, N. Flores-Herr, A. Löhden, I. Konya, D. Baum, A. Bergholz, D. Schönfuß, C. Fey, J. Osterhoff, J. Waitelonis, H. Sack, R. Köhler, P. NdjikiNya, CONTENTUS—technologies for next generation multimedia libraries, Multimedia Tools Appl. 63 (2) (2013) 287–329.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS