Evaluation of conversational agents for aerospace domain

Ying Hsang Liu, Alexandre Arnold, Gérard Dupont, Catherine Kobus, François Lancelot

Publikation: Bidrag til tidsskriftKonferenceartikelForskningpeer review

7 Downloads (Pure)


The use of conversational agents within the aerospace industry offers quick and concise answers to complex situations. The aerospace domain is characterized by products and systems that are built over decades of engineering to reach high levels of performance within complex environments. Current development in conversational agents can leverage the latest retrieval and language model to refine the system's question-answering capabilities. However, evaluating the added-value of such a system in the context of industrial applications such as pilots in a cockpit is complex. This paper describes how a conversational agent is implemented and evaluated, with particular references to how state-of-the-art technologies can be adapted to the domain specificity. Preliminary findings of a controlled user experiment suggest that user perception of the usefulness of the system in completing the search task and the system's responses to the relevance of the topic are good predictors of user search performance. User satisfaction with the system's responses may not be a good predictor of user search performance.

TidsskriftCEUR Workshop Proceedings
Antal sider9
StatusUdgivet - 2020
Begivenhed1st Joint Conference of the Information Retrieval Communities in Europe, CIRCLE 2020 - Samatan, Gers, Frankrig
Varighed: 6. jul. 20209. jul. 2020


Konference1st Joint Conference of the Information Retrieval Communities in Europe, CIRCLE 2020
BySamatan, Gers

Fingeraftryk Dyk ned i forskningsemnerne om 'Evaluation of conversational agents for aerospace domain'. Sammen danner de et unikt fingeraftryk.