Content area

Abstract

Translate

The development of conversational AI assistants is an iterative process with multiple components. As such, the evaluation and continual improvement of these assistants is a complex and multifaceted problem. This paper introduces the challenges in evaluating and improving a generative AI assistant for enterprises, which is under active development, and how we address these challenges. We also share preliminary results and discuss lessons learned.

Details

1009240
Subject
Conversational artificial intelligence;
Generative artificial intelligence
Identifier / keyword
Human-Computer Interaction
URL
http://arxiv.org/abs/2407.12003
Title
Evaluation and Continual Improvement for an Enterprise AI Assistant
Author
Maharaj, Akash V; Qian, Kun; Bhattacharya, Uttaran; Fang, Sally; Galatanu, Horia; Garg, Manas; Hanessian, Rachel; Kapoor, Nishant; Russell, Ken; Vaithyanathan, Shivakumar; Li, Yunyao
Publication title
arXiv.org; Ithaca
Publication year
2024
Publication date
Dec 8, 2024
Section
Computer Science
Publisher
Cornell University Library, arXiv.org
Source
arXiv.org
Place of publication
Ithaca
Country of publication
United States
University/institution
Cornell University Library arXiv.org
Publication subject
Biology, Statistics, Physics, Mathematics, Engineering--Electrical Engineering, Computers--Computer Engineering, Business And Economics--Banking And Finance
e-ISSN
2331-8422
Source type
Working Paper
Language of publication
English
Document type
Working Paper
Publication history
 
 
Online publication date
2024-12-10
Milestone dates
2024-06-15 (Submission v1); 2024-12-08 (Submission v2)
Publication history
 
 
   First posting date
10 Dec 2024
ProQuest document ID
3082383924
Document URL
https://www.proquest.com/working-papers/evaluation-continual-improvement-enterprise-ai/docview/3082383924/se-2?accountid=208611
Full text outside of ProQuest
http://arxiv.org/abs/2407.12003
Copyright
© 2024. This work is published under http://creativecommons.org/licenses/by-nc-sa/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Last updated
2024-12-11
Database
2 databases
  • ProQuest One Academic
  • ProQuest One Academic