Exploring Inter Tagger Consistency Measures

Kipp, Margaret EI Exploring Inter Tagger Consistency Measures., 2009 . In 20th Annual SIG/CR Classification Research Workshop, American Society for Information Science and Technology, Vancouver, BC, 6-11 November 2009. (Unpublished) [Conference poster]

Preview

PDF sigcrposter2009.pdf Download (329kB) | Preview

English abstract

Kipp and Campbell (2006) examined tags assigned to the same URL in del.icio.us and determined that MDS and frequency graphs showed clusters of related terms as well as divergences between synonyms. Professional indexers too exhibit convergence and divergence in indexing behaviour, which has been measured in inter-indexer consistency studies. Leonard (1977) and Markey (1984) examined the results of multiple inter-indexer consistency studies examining not only the levels of inconsistency which varied widely but also the level of indexing exhaustivity (number of terms assigned to each document), method of collecting indexing data and vocabulary size. The majority of inter-indexer consistency studies show high levels of inconsistency between indexers (Leonard 1977; Markey 1984). While inter-indexer consistency studies have traditionally compared the indexing terms used by a small group of indexers, it is possible to adapt some of the more common measures to be used with large groups of indexers. A number of measures were examined in this study to determine which measures provide the most ability to distinguish between different indexers. This study used Salton's Cosine measure, the Jaccard measure--also known as Hooper and Rolling's measures (Markey 1984), Wolfram and Olsen's Inter-indexer Consistency Density (Wolfram and Olsen 2007) and a Pairwise Jaccard measure (compares all indexers to each other without the need for a centroid or known good set of index terms). This study is part of a larger study examining measures of convergence and divergence in tagging systems. One goal of the larger study is to examine different ways of analysing tag data to see which methods provide the most useful analyses of the structures which develop in tagging. By calculating a number of different inter-indexer consistency measures it may be possible to make distinctions between tag lists to provide predictive analysis of tagging patterns.

Item type:	Conference poster
Keywords:	tagging, citeulike, intertagger consistency
Subjects:	I. Information treatment for information services > ID. Knowledge representation. I. Information treatment for information services > IC. Index languages, processes and schemes.
Depositing user:	Margaret E. I. Kipp
Date deposited:	04 Dec 2009
Last modified:	02 Oct 2014 12:15
URI:	http://hdl.handle.net/10760/13794

References

Kipp, Margaret E.I. and Campbell, D. Grant. 2006. Patterns and Inconsistencies in Collaborative Tagging Systems: An Examination of Tagging Practices. Proceedings of the 2006 Annual Meeting of the American Society for Information Science and Technology, Austin, November 3-8, 2006. http://eprints.rclis.org/archive/00008315/

Leonard, L. 1977. Inter-indexer consistency studies, 1954-1975: a review of the literature and summary of study results. Occasional papers (University of Illinois at Urbana-Champaign. Graduate School of Library Science) 131. http://hdl.handle.net/2142/3885

Markey, Karen. 1984. Interindexer Consistency Tests: a Literature Review and Report of a Test of Consistency in Indexing Visual Materials. Library and Information Science Research 6(2): 155-177.

Wolfram, Deitmar and Olson, Hope. 2007. A method for comparing large scale inter-indexer consistency using IR modeling. Proceedings of the 35th conference of the Canadian Association for Information Science, Montreal, QC, May 10-12, 2007. http://www.caisacsi.ca/proceedings/2007/wolfram_2007.pdf

Downloads

Actions (login required)

View Item