使用Player FM应用程序离线!
ATGthePodcast 220 - Evaluating AI Tools For Researchers: A Guide For Libraries
Manage episode 381321849 series 1327300
Audio from the 2022 Charleston Conference from a Session titled "Evaluating AI Tools For Researchers: A Guide For Libraries.”
This session was presented by Michael Upshall, Consultant, Former Publisher, and Igor Brbre, Information Specialist, NHS Healthcare Improvement, Scotland, and research librarian.
In this session, Michael and Igor outline an assessment framework for evaluating new AI-based tools, from an evaluation of the corpus to being able to measure accuracy and effectiveness, as well as being able to identify biases and inequalities.
The evaluation framework is exemplified in a case study including a measured trial of some of these tools, comparing human and machine-facilitated approaches to the research process, and comparing time taken and quality of results, using data from a health library. Until now, much of the help provided by libraries in the form of fact sheets and how-to guides has been evaluated only subjectively; this case studies aims to show a better way to evaluate these tools, using statistically valid samples plus feedback from users to identify how widely used and how successful these tools are, in the hopes of being better equipped to evaluate new AI utilities for research and submission.
Video of the presentation available at: https://youtu.be/ifZ-xQKw-oY?si=CQ3Yflo-zhJV_VMO
Social Media:
https://www.linkedin.com/in/mupshall/
Twitter:
Keywords: #AI, #AITools, #AIServices, #research, #knowledge, #scholcomm, #collaboration,#engagement, #problemsolvers, #publishing, #libraries, #librarians, #information, #ChsConf, #LibrariesAndVendors, #LibrariesAndPublishers,
249集单集
Manage episode 381321849 series 1327300
Audio from the 2022 Charleston Conference from a Session titled "Evaluating AI Tools For Researchers: A Guide For Libraries.”
This session was presented by Michael Upshall, Consultant, Former Publisher, and Igor Brbre, Information Specialist, NHS Healthcare Improvement, Scotland, and research librarian.
In this session, Michael and Igor outline an assessment framework for evaluating new AI-based tools, from an evaluation of the corpus to being able to measure accuracy and effectiveness, as well as being able to identify biases and inequalities.
The evaluation framework is exemplified in a case study including a measured trial of some of these tools, comparing human and machine-facilitated approaches to the research process, and comparing time taken and quality of results, using data from a health library. Until now, much of the help provided by libraries in the form of fact sheets and how-to guides has been evaluated only subjectively; this case studies aims to show a better way to evaluate these tools, using statistically valid samples plus feedback from users to identify how widely used and how successful these tools are, in the hopes of being better equipped to evaluate new AI utilities for research and submission.
Video of the presentation available at: https://youtu.be/ifZ-xQKw-oY?si=CQ3Yflo-zhJV_VMO
Social Media:
https://www.linkedin.com/in/mupshall/
Twitter:
Keywords: #AI, #AITools, #AIServices, #research, #knowledge, #scholcomm, #collaboration,#engagement, #problemsolvers, #publishing, #libraries, #librarians, #information, #ChsConf, #LibrariesAndVendors, #LibrariesAndPublishers,
249集单集
所有剧集
×欢迎使用Player FM
Player FM正在网上搜索高质量的播客,以便您现在享受。它是最好的播客应用程序,适用于安卓、iPhone和网络。注册以跨设备同步订阅。