The paradox of expertise: Is the Wikipedia Reference Desk as good as your library?

Shachaf, Pnina The paradox of expertise: Is the Wikipedia Reference Desk as good as your library? Journal of Documentation, 2009, vol. 65, n. 6, pp. 977-996. [Journal article (Paginated)]

[img]
Preview
 Text
Paradox of expertise-final.pdf
Download (843kB) | Preview

English abstract

Purpose: This study aims to examine answers’ quality on the Wikipedia Reference Desk, and to compare it with library reference services. It examines whether Wikipedia volunteers outperform expert reference librarians and exemplify the paradox of expertise. Design/Methodology/Approach The study applied content analysis to a sample of 434 messages (77 questions and 357 responses) from the Wikipedia Reference Desk and focused on three SERVQUAL quality variables: reliability (accuracy, completeness, verifiability), responsiveness, and assurance. Findings The study reports that on all three SERVQUAL measures quality of answers produced by the Wikipedia Reference Desk is comparable with that of library reference services. Research limitations/Implications The collaborative social reference model matched or outperformed the dyadic reference interview and should be further examined theoretically and empirically. The generalizability of the findings to other similar sites is questionable. Practical implications Librarians and library science educators should examine the implications of the social reference on the future role of reference services. Originality value This study is the first to: 1) examine the quality of the Wikipedia reference desk; 2) extend research on Wikipedia quality; 3) use SERVQUAL measures in evaluating Q&A sites; and 4) compare Q&A sites with traditional reference services.

Item type: Journal article (Paginated)
Keywords: Reference services, Wikipedia, quality, reliability, social reference, Q&A sites.
Subjects: A. Theoretical and general aspects of libraries and information. > AB. Information theory and library theory.
B. Information use and sociology of information > BA. Use and impact of information.
B. Information use and sociology of information > BC. Information in society.
B. Information use and sociology of information > BG. Information dissemination and diffusion.
H. Information sources, supports, channels. > HP. e-resources.
H. Information sources, supports, channels. > HQ. Web pages.
H. Information sources, supports, channels. > HS. Repositories.
I. Information treatment for information services > IJ. Reference work.
L. Information technology and library technology > LC. Internet, including WWW.
Depositing user: Amanda Ferrara
Date deposited: 07 Oct 2013 15:37
Last modified: 02 Oct 2014 12:28
URI: http://hdl.handle.net/10760/20329

References

Adamic, L. A., Zhang, J., Bakshy, E. and Ackerman, M.S. (2008), “Knowledge sharing and Yahoo! Answers: Everyone knows something”, in WWW '08: Proceeding of the 17th international conference on World Wide Web, pages 665-674, ACM, New York, NY.

Agosto, D.E. (2002), “Bounded rationality and satisficing in young people's web-based decision making”, Journal of the American Society for Information Science and Technology, Vol. 53 No. 1, pp. 16-27.

Agichtein, E., Castillo, C., Donato, D., Gionides, A. and Mishne, G. (2008). “Finding high-quality content in social media”, in WSDM '08: Proceedings of the international conference on Web search and web data mining, pages 183-194, ACM, New York, NY.

Arnold, J. and Kaske, N. (2005), “Evaluating the quality of a chat service”, Portal: Libraries and the Academy, Vol. 5 No. 2, pp. 177-193.

Bawden, D. (2007), "Facing the educational future", Information Research, Vol. 12 No. 4, paper colise01, available at http://InformationR.net/ir/12-4/colis/colise01 (accessed 15 Sept 2008).

Bian, J., Liu, Y., Agichtein, E. and Zha, H. (2008), “Finding the right facts in the crowd: factoid question answering over social media”, in WWW '08: Proceeding of the 17th international conference on World Wide Web, pages 467-476, ACM, New York, NY.

Cahill, K. (2007) “Worth the price? : Virtual reference, global knowledge forums, and the demise of Google Answers”, Journal of Library Administration, Vol. 46 No. 3/4, pp. 73-86.

Carter, D.S. and Janes, J. (2000), “Unobtrusive data analysis of digital reference questions and service at the Internet Public Library: An exploratory study”, Library Trends, Vol. 49 No. 2, pp. 251-265.

Chen, W., Zeng, Q. and Wenyin, L. (2006), “A user reputation model for a user-interactive question answering system”, in Proceedings of the Second International Conference on Semantics, Knowledge, and

Grid, page 40, IEEE Computer Society, Washington, DC.

Dom, B. and Paranjpe, D. (2008), “A bayesian technique for estimating the credibility of question answerers”, in SIAM International Conference on Data Mining, pages 399-409, SIAM, Atlanta, GA.

Emigh, W. and Herring, S. C. (2005), “Collaborative authoring on the Web: A genre analysis of online encyclopedias”, in Proceedings of the Thirty-Eighth Hawai'i International Conference on System Sciences, IEEE Computer Society, Washington, DC.

Gazan, R. (2006), “Specialists and synthesists in a question answering community”, in Proceedings of ASIS&T Conference, Vol. 43, pp. 777–788.

Gazan, R. (2008), “Seekers, sloths and social reference: Homework questions submitted to a question-answering community”, New Review of Hypermedia & Multimedia, Vol. 13 No. 2, pp. 239-248.

Giles, J. (2005), “Internet encyclopedias go head to head”, Nature, December 14, 2005, available at: http://www.nature.com/news/2005/051212/full/438900a.html (accessed 19 August 2008).

Guidelines. (September 3, 2007), in Wikipedia, the free encyclopedia, available at: http://en.wikipedia.org/wiki/Wikipedia:Reference_desk/Guidelines, accessed 3 September 2007).

Harper, F.M., Raban, D., Rafaeli, S. and Konstan, J.A. (2008), “Predictors of answer quality in online Q&A sites”, in CHI '08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, pages 865-874, ACM, New York, NY.

Hernon, P. and Calvert, P. (2005), “E-service quality in libraries: Exploring its feature and dimensions”, Library & Information Science Research, Vol. 27, pp. 377-404.

Hernon, P. and McClure, C.R. (1986), “Unobtrusive reference testing: The 55 percent rule”, Library Journal, Vol. 111, pp. 37-41.

Janes, J.W. and Mon, L. (2006), “The thank you study: User satisfaction with digital reference services”, available at: http://webjunction.org/c/document_library/get_file?folderId=441426&name=DLFE-11969.pdf (accessed 14 August 14 2008).

Johnson, P.E. (1983), “What kind of expert should a system be?" The Journal Of Medicine And Philosophy, Vol. 8 No. 1, pp. 77-97.

Johnson, R. (2003), “Integrating methodologists into teams of substantive experts”, Studies in Intelligence, Vol. 47 No. 1, article 06, available at: http://www.au.af.mil/au/awc/awcgate/cia/reducing_analytic_error.htm (accessed 4 October 2008).

Jurczyk, P. and Agichtein, E. (2007a), “Discovering authorities in question answer communities by using link analysis”, in CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, pages 919-922, ACM, New York, NY.

Jurczyk, P. and Agichtein, E. (2007b), “Hits on question answer portals: Exploration of link analysis for author ranking”, Poster session presented at the Annual ACM Conference on Research and Development in Information Retrieval, ACM, Amsterdam, Netherlands.

Kaske, N. and Arnold, J. (2002), “An unobtrusive evaluation of online real time library reference services”, available at: http://www.lib.umd.edu/groups/digref/kaskearnoldunobtrusive.html (accessed 14 September 2008).

Keen, E. (2008), The Cult of the Amateur: How Today’s Internet is Killing Our Culture. Doubleday/Currency, New York, NY.

Kim, S., Oh, S.J. and Oh, S. (2007), “Best-answer selection criteria in a social Q&A site from the user-oriented relevance perspective”, Poster session presented at the ASIS&T 2007 Annual Meeting, Milwaukee, WI.

Korfiatis, N., Poulous, M. and Bokos, G. (2006), “Evaluating authoritative sources using social networks: An insight from Wikipedia. Online Information Review, Vol. 30 No. 3, pp. 252-262, available at: http://www.korfiatis.info/papers/OISJournal_final.pdf (accessed 5 Apil 2007).

Lankes, D.R. (2008), “Credibility on the Internet: Shifting from authority to reliability", Journal of Documentation, Vol. 64 No. 5, pp. 667-686.

Mansourian, Y. and Ford, N. (2007), “Search persistence and failure on the web: A "bounded rationality" and "satisficing" analysis”, Journal of Documentation, Vol. 63 No. 5, pp. 680-701.

Mon, L. and Janes, J.W. (2008), “The thank you study: User feedback in e-mail thank you messages”, Reference & User Services Quarterly, Vol. 46 No. 4, pp. 53-59.

Noguchi, Y. (2006), “Web searches go low-tech: You ask, a person answers”, Washington Post, August 16, 2006, pp. A01, available at: http://www.washingtonpost.com/wp-dyn/content/article/2006/08/15/AR2006081501142.htm (accessed 20 August 2008).

O'Neill M., Wright C. and Fitz F. (2001), “Quality evaluation in on-line service environments: An application of the importance-performance measurement technique”, Managing Service Quality, Vol. 11 No. 6, pp. 402-417.

O’Neill, N. (2007), “Chacha, Yahoo!, and Amazon”, Searcher, Vol. 15 No. 4, pp. 7-11.

Parasuraman, A. (1985), “A conceptual model of service quality and its implications for future research”, Journal of Marketing, Vol. 49, pp. 41-50.

Parasuraman, A., Zeithaml, V.A. and Berry, L.L. (1988), “SERVQUAL: A multiple item scale for measuring consumer perceptions of service quality”, Journal of Retailing, vol. 64 No. 1, pp. 12-40.

Radford, M.L. and Connaway, L.S. (2008), “Getting better all the time: Improving communication and accuracy in virtual reference”, Paper presented at Reference Renaissance: Current and Future Trends, Denver, CO, available at: http://www.oclc.org/research/projects/synchronicity/ppt/refren08.ppt (accessed 19 September 2008)

Richman, D. (2007), “Social search comes of age”, Information Outlook, Vol. 11 No. 8, pp. 18-25.

Rosenzweig, R. (2006), “Can history be open source? Wikipedia and the future of the past”, Journal of American History, Vol. 93 No. 1, pp. 117-146.

Saxton, M.L. and Richardson, J.V. (2002), Understanding reference transactions: Transforming an art into a science, Academic Press, San Diego, CA.

Shachaf, P. and Horowitz, S. (2008), “Virtual reference service evaluation: Adherence to RUSA behavioral guidelines and IFLA digital reference guidelines”, Library & Information Science Research, Vol. 30 No. 2, pp. 122-137.

Shachaf, P., Oltmann, M.S. and Horowitz, S. (2008), “Service equality in virtual reference”, Journal of the American Society for Information Science and Technology, Vol. 59 No. 4, pp. 535-550.

Shachaf, P. and Shaw, D. (2008), “Bibliometric analysis to identify core reference sources of virtual reference transactions”, Library & Information Science Research, Vol. 30 No. 4, pp. 291-297.

Shah, C., Oh, J.S., and Oh, S. (2008). Exploring characteristics and effects of user participation in online social Q&A sites. First Monday, 13, 9. Available at: http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/viewArticle/2182/2028 (accessed March 19, 2009).

Stvilia, B., Twidale, M.B., Smith, L.C. and Gasser, L. (2005), “Assessing information quality of a community-based encyclopedia”, in Proceedings of the International Conference on Information Quality - ICIQ 2005, pages 442-454, available at: http://mailer.fsu.edu/~bstvilia/papers/quantWiki.pdf (accessed 19 August 2008).

Surowiecki, J. (2004), The Wisdom of the Crowds, Anchor Books, New York, NY.

Tapscott, D. and Williams, A. D. (2007), “Wikinomics: How Mass Collaboration Changes Everything, Penguin Group Inc, New-York, NY.

Tetlock, P.E. (2005), Expert Political Judgment: How Good is it? How Can We Know? Princeton University Press, Princeton, NJ.

Viégas, F., Wattenberg, M. and Dave, K. (2004), “Studying cooperation and conflict between authors with history flow visualizations”, in Proceedings of the 2004 Computer-Human Interaction, Vol. 6 No. 1, pp. 575-582.

Viégas, F.B., Wattenberg, M., Kriss, J. and van Ham, F. (2007), “Talk before you type: Coordination in Wikipedia”, in Proceedings of the Forty Hawai'i International Conference on System Sciences, January 7-10, 2007, Big Island Hawaii, IEEE Press, Los Alamitos, CA.

Voss, C.A. (2003), “Rethinking paradigms of service: Service in a virtual environment”, International Journal of Operations & Production Management, Vol. 23 No. 1, pp. 88-104.

Ward, D. (2005), “Why users choose chat?: A survey of behavior and motivations,” Internet Reference Services Quarterly, Vol. 10 No. 1, pp. 29-46.

Weinberger, D. (2007), Everything is Miscellaneous: The Power of the New Digital Disorder, Henry Holt & Co., New York, NY.

Wikipedia Reference Desk. (2008), in Wikipedia, the free encyclopedia, available at: http://en.wikipedia.org/wiki/Wikipedia:Reference_desk (accessed 15 September 2008).

Willinsky, J. (2007), “What open access research can do for Wikipedia”, First Monday, Vol. 12 No. 3), available at: http://www.firstmonday.org/issues/issue12_3/ (accessed 15 September 2008).

Yang, Z., Jun, M. and Peterson, R.T. (2004), “Measuring costumer perceived online service quality: Scale development and managerial implications”, International Journal of Operations & Production Management, Vol. 24 No. 11/12, pp. 1149-1174.

Yu, L., Hong, Q., Gu, S. and Wang, Y. (2008), “An epistemological critique of gap theory based library assessment: The case of SERVQUAL”, Journal of Documentation, Vol. 64 No. 4, pp. 511-551.

Zhang, J., Ackerman, M.S., Adamic, L. and Nam, K.K. (2007), “QuME: A mechanism to support expertise finding in online help-seeking communities”, in UIST '07: Proceedings of the 20th annual ACM symposium on User interface software and technology, pages 111-114, ACM, New York, NY.

Zipf, G.K. (1949), Human Behaviour and the Principle of Least Effort, Addison-Wesley, Reading, MA.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS