ChatGPT Search as a tool for scholarly tasks: evolution or devolution?

Font-Julián, Cristina I., Orduña-Malea, Enrique and Codina, Lluís ChatGPT Search as a tool for scholarly tasks: evolution or devolution? Infonomy, 2024, vol. 2, n. 5. [Journal article (Unpaginated)]

[thumbnail of Font-Orduna-Codina-ChatGPT-Search-as-a-tool-for-scholarly-tasks.pdf]
Preview
 Text
Font-Orduna-Codina-ChatGPT-Search-as-a-tool-for-scholarly-tasks.pdf - Published version
Available under License Creative Commons Attribution.
Download (585kB) | Preview

English abstract

ChatGPT Search was launched on October 31 by OpenAI as a new AI-powered search engine. Among its features, it stands out for its ability to retrieve information from various online sources, including scholarly databases, which potentially allows the use of this tool for academic tasks, both quantitative and qualitative. To test its features, five academic tasks are designed: two quantitative (collecting hit count estimates from Google Search and scraping bibliometric indicators from ResearchGate); two qualitative tasks (performing a narrative synthesis of an academic topic and generating a brief academic author profile), and a mixed task (identifying, collecting and describing a list of publications from Google Scholar Profiles). The results show the inability of ChatGPT Search to conduct quantitative tasks correctly, fabricating the results (hallucination). Qualitative tasks are performed with better results; however, errors are detected, which prevent recommending the tool without manual analysis and refinement. Finally, the ability to generate links to scientific publications can open up competition among academic sites to be mentioned in the ChatGPT Search responses, giving rise to Academic Generative Engine Optimization (A-GEO).

Item type: Journal article (Unpaginated)
Keywords: Generative artificial intelligence; AI; Link analysis; Web search; Search engines; Narrative synthesis; Information retrieval; ChatGPT Search; Scholarly tasks; Academic tasks; Quantitative vs. Qualitative; Academic Generative Engine Optimization (A-GEO).
Subjects: B. Information use and sociology of information > BC. Information in society.
L. Information technology and library technology > LC. Internet, including WWW.
L. Information technology and library technology > LP. Intelligent agents.
Depositing user: Dr. Nicoleta-Roxana Dinu
Date deposited: 29 Dec 2024 18:08
Last modified: 29 Dec 2024 18:08
URI: http://hdl.handle.net/10760/46289

References

Aggarwal, Pranjal; Murahari, Vishvak; Rajpurohit, Tanmay; Kalyan, Ashwin; Narasimhan, Karthik; Deshpande, Ameet (2024). Geo: Generative engine optimization. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 5-16). New York (USA): ACM. https://arxiv.org/pdf/2311.09735

Bains, Callum (2024). The chatbot optimisation game: Can we trust AI web searches? The Guardian, 3 November. https://www.theguardian.com/technology/2024/nov/03/the-chatbot-optimisation-game-can-we-trust-ai-web-searches

Lopezosa, Carlos; Goyanes, Manuel; Codina, Lluís (2024) Acelerando la investigación cualitativa con inteligencia artificial: una guía práctica para el diseño, desarrollo y ejecución de investigación con entrevistas. Barcelona: Universitat de Barcelona. http://hdl.handle.net/2445/211948

Lopezosa, Carlos; Rovira, Cristòfol; Codina, Lluís (2024). La IA generativa y su aplicación al SEO, SEM, analítica web y diseño UX. Barcelona: Universitat Pompeu Fabra, 92 pp. https://doi.org/10.31009/cr.2024.11

Orduña-Malea, Enrique; Alonso-Arroyo, Adolfo (2017). Cybermetric techniques to evaluate organizations using web-based data. Oxford: Chandos Publishing. https://doi.org/10.1016/C2016-0-00399-6

Thelwall, Mike (2010). Webometrics: Emergent or doomed? Information Research: An International Electronic Journal, v. 15, n. 4. https://informationr.net/ir/15-4/colis713.html

Thelwall, Mike; Stuart, David (2006). Web crawling ethics revisited: Cost, privacy, and denial of service. Journal of the American Society for Information Science and Technology, v. 57, n. 13, pp. 1771–1779. https://doi.org/10.1002/asi.20388

Urbano, Cristóbal (2024). Los chatbots de IA

generativa como fuentes de tráfico web: aproximación desde la analítica web de un portal de revistas. Anuario ThinkEPI, v.18. https://doi.org/10.3145/thinkepi.2024.e18a31

Wan, Alexander; Wallace, Eric; Klein, Dan (2024). What evidence do language models find convincing? arXiv preprint. https://arxiv.org/pdf/2402.11782


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS