A framework for assessing the impact of units of scholarly communication based on OAI-PMH harvesting of usage information
(2005) A framework for assessing the impact of units of scholarly communication based on OAI-PMH harvesting of usage information. Delivered at CERN workshop on Innovations in Scholarly Communication (OAI4), Geneva (Switzerland). Presentation.
Full text available as: |
Abstract
The wide-spread implementation of institutional repositories (IR), digital libraries, preprint services, and open access journals has dramatically changed the communication options that are available to scholars.
At the same time, scholarship itself is becoming digital, thereby fundamentally extending the notion of a unit of
scholarly communication beyond journal papers to include multimedia files, data sets, simulations, visualizations, etc. Meanwhile, the evaluation of scholarly performance
remains bound to the use of citation data derived from a subset of all available communication channels (pre-selected journals), and an ever decreasing subset of all communicated units (journal papers). Clearly, there is a need for frameworks that allow measuring scholarly activity and its impact in the context of this new reality.
We discuss the architecture of a system that is being developed at the Los Alamos National Laboratory that aims at determining impact and prestige rankings on the basis of aggregated usage data. This system relies on two key components. First, an architecture that allows to OAI-PMH harvest, and hence aggregate, usage logs from various scholarly communication venues. For interoperability, usage logs are expressed as XML documents that are compliant with the ContextObject of the OpenURL Standard. Second, a set of social network methods to determine impact and prestige from the temporal patterns detected in the aggregated usage data. The proposed solution can be deployed on top of any type of scholarly communication channel, and can take into account the use of scholarly communication units of all types.
We discuss recent results which indicate that, when applied to articles and journals, the resulting impact rankings correlate significantly with the Institute for Scientific Information's Impact Factor, but highlight different aspects of publication status and can thus form the basis of a more comprehensive assessment of scholarly impact. We speculate on how an open, freely accessible system for the evaluation of science relying on widely aggregated usage data can be applied to a wider range of scholarly communication processes then is presently the case, and can ultimately liberate the scientific community from the limitations and distortions caused by the existing singular focus on proprietary, citation based science evaluation mechanisms.
| Keywords: | Scholarly communication, OAI-PMH harvesting, OpenURL, social network methods, evaluation of science, citation data, impact factors |
|---|---|
| Subjects: | I. Information treatment for information services > ID. Knowledge representation. I. Information treatment for information services > IE. Data and metadata structures. I. Information treatment for information services > IF. Information transfer: protocols, formats, techniques. |
| ID Code: | 6076 |
| Deposited By: | Italian Staff, E-LIS |
| Deposited On: | 22 April 2006 |
| Alternative Locations: | http://indico.cern.ch/materialDisplay.py?contribId=48&sessionId=1&materialId=slides&confId=0514, http://indico.cern.ch/materialDisplay.py?contribId=48&sessionId=1&materialId=video&confId=0514 |
| All fields: | Show all fields |
Archive Staff Only: edit this record