Kipp, Margaret E.I. Tagging and Findability: Do Tags Help Users Find Things?, 2007 . In Annual General Meeting of the American Society for Information Science and Technology, Milwaukee, Wisconsin, USA, October 19-24, 2007. (Unpublished) [Conference poster]
|
PDF
asist2007poster.pdf Download (97kB) | Preview |
English abstract
Background The question of findability is an old one and led directly to the creation of cataloguing and classification systems for the organisation of knowledge. However, such systems have not proven to be truly scalable when dealing with digital information and especially information on the web. Can the user created categories and classification schemes of tagging be used to enhance findability in these new environments? Much speculation has been advanced on the subject but so far no empirical studies have been done. This study looks at the use of tags as an aid to findability. In essence, this is an examination of some of the questions of classic information retrieval research in a new context. Proponents of tagging often suggest that tags could provide at worst an entry vocabulary for traditional classification systems and at best a complete replacement for such systems. One method for judging the usefulness of a classification system for enabling retrieval is to perform an information retrieval study on the system with volunteer searchers. Many such experiments have been conducted in the library science literature and increasingly these studies are performed on web search engines as more and more users search the web first. Studies by Leighton and Srivastava (1999), Clarke and Willet (1997), Su (2003b) all evaluated a variety of search engines using standard evaluation methods from earlier non web information retrieval systems such as recall and precision, but many also suggested alternative methods for evaluating the effectiveness of a search engine given that recall and precision cannot truly be calculated. (Tang and Sun 2003; Vaughan 2004) A key component in all modern information retrieval studies is the relevance judgements of users of the retrieval system. (Cosijn and Ingwersen 2000; Tang and Sun 2003; Oppenheim et al. 2000) Therefore, this study would place a great emphasis on the relevance judgements of the participants. As noted above, relevance judgements are often used to enable the calculation of metrics for evaluating the effectiveness of information retrieval systems. In all studies examining information retrieval systems using keyword or index term searches, there is an implicit evaluation of the effectiveness of classification terms. In evaluating classification terms, it is important to evaluate not only the retrieval effectiveness of the search term, in terms of how many relevant documents were retrieved, but also in terms of how long it took the user to think of using this term in this context and whether or not the user thought the term was appropriate or useful for the document. One way to examine the potential uses of tags in the search process would be to compare the search experience between social bookmarking tools and other methods of information retrieval such as retrieval via controlled vocabulary or retrieval via free text search. Research Questions Do tags appear to enhance findability? Do users feel that they have found what they are looking for? How do users find searching social bookmarking sites compared to searching more classically organised sites? Do users think that tags assigned by other users are more intuitive? Do tagging structures facilitate information retrieval? How does this compare to traditional structures of supporting information retrieval? Methodology and Analysis In order to compare the usability of tags versus traditional classification schemes, a study will be conducted using volunteer searchers. These searchers would be asked to search an electronic journal database and a social bookmarking site for information on a specific topic. Screen capture software, a think aloud protocol and an exit interview will be used to capture the impressions of the users when faced with traditional classification or user tags and their usefulness in the search process. While information concerning the usability of the systems themselves for searching may be of interest, data collection will be focused on a comparison of the terms entered by the participants. A sample size of 20 participants for the study should allow for the collection of data until saturation is reached, in other words until no further new information is being collected. Since this is an exploratory study, 20 participants should provide a good selection for the collection of data. Each participant will search for information using both the traditional on-line database with assigned descriptors and a social bookmarking site. This study will ask users to use pubmed (an on-line database) in comparison to citeulike (a social bookmarking tool) to find academic articles on given topics. Participants will be asked to perform the searches in the order specified so that their use of a social bookmarking site first versus an on-line database can be randomised. Participants will be divided into two groups. One group will search the social bookmarking tool first while the other group will use the on-line database first. Participants will be assigned a topic for which they will search for articles. These topics will be articulated as short paragraphs describing an information need. Participants will select their own keywords for both the on-line database and the social bookmarking tools and then provide relevance judgements of the results. Initially, participants will be asked to provide a list of words they consider relevant as an initial search set. Each search will produce a result set of bibliographic information. The participant will be asked to select the top 10 articles from the result set and assign a relevance score to the article based on an examination of the available information. The available information may include the title, tags and abstract as well as the full text of the article via links to an on-line database. At the end of each search, participants will be asked to make a list of what terms they would now use if asked to search for this information again. Participants will not have access to their initial set of search terms at this time to eliminate the learning effect. Three sets of data will be available for analysis: sets of initial and final keywords selected by the user, the recording of the search session and think aloud, and recorded exit interviews after the search session. Each set of data can be analysed to examine user impressions of the search process from the perspective of the keywords (tags or index terms respectively). Both traditional and newly suggested measures for evaluating information retrieval systems will be used to compare the two systems on standard efficiency measures. Additionally, keywords and tags chosen by users will be compared and examined to see how they are related (or not). Keywords and tags assigned to articles judged relevant by the users will be compared to those used for articles judged to be less relevant or not relevant. It is expected that this preliminary study will provide some insights into user's choices of preliminary keywords for searching. One of the questions that users will be asked is to describe how they selected their initial search terms. They will also be asked to compare their initial search terms to their final search terms.
| Item type: | Conference poster |
|---|---|
| Keywords: | tagging, findability, information retrieval |
| Subjects: | I. Information treatment for information services > ID. Knowledge representation. |
| Depositing user: | Margaret E. I. Kipp |
| Date deposited: | 12 Jun 2008 |
| Last modified: | 02 Oct 2014 12:09 |
| URI: | http://hdl.handle.net/10760/10574 |
References
Downloads
Downloads per month over past year
Actions (login required)
 |
View Item |
Facebook
Twitter
LinkedIn
RSS