Writer-Defined AI Personas for On-Demand Feedback Generation
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
Abstract:
Compelling writing is tailored to its audience. This is challenging, as writers may struggle to empathize with readers, get feedback in time, or gain access to the target group. We propose a concept that generates on-demand feedback, based on writer-defined AI personas of any target audience. We explore this concept with a prototype (using GPT-3.5) in two user studies (N=5 and N=11): Writers appreciated the concept and strategically used personas for getting different perspectives. The feedback was seen as helpful and inspired revisions of text and personas, although it was often verbose and unspecific. We discuss the impact of on-demand feedback, the limited representativity of contemporary AI systems, and further ideas for defining AI personas. This work contributes to the vision of supporting writers with AI by expanding the socio-technical perspective in AI tool design: To empower creators, we also need to keep in mind their relationship to an audience.
Our text editor Impressona supports writers with generated feedback on their draft: While working on a text (A), writers can define AI personas that represent their target readers (B). Writers can select any part of their text and click on a persona button (C) to receive feedback from the perspective of that persona. This feedback is generated with a Large Language Model (GPT-3.5) by prompting it with the provided persona information. (B) and (C) show different tab states of the same sidebar, not two sidebars.
BibTeX:
@inproceedings{10.1145/3613904.3642406,
author = {Benharrak, Karim and Zindulka, Tim and Lehmann, Florian and Heuer, Hendrik and Buschek, Daniel},
title = {Writer-Defined AI Personas for On-Demand Feedback Generation},
year = {2024},
isbn = {9798400703300},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3613904.3642406},
doi = {10.1145/3613904.3642406},
abstract = {Compelling writing is tailored to its audience. This is challenging, as writers may struggle to empathize with readers, get feedback in time, or gain access to the target group. We propose a concept that generates on-demand feedback, based on writer-defined AI personas of any target audience. We explore this concept with a prototype (using GPT-3.5) in two user studies (N=5 and N=11): Writers appreciated the concept and strategically used personas for getting different perspectives. The feedback was seen as helpful and inspired revisions of text and personas, although it was often verbose and unspecific. We discuss the impact of on-demand feedback, the limited representativity of contemporary AI systems, and further ideas for defining AI personas. This work contributes to the vision of supporting writers with AI by expanding the socio-technical perspective in AI tool design: To empower creators, we also need to keep in mind their relationship to an audience.},
booktitle = {Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems},
articleno = {1049},
numpages = {18},
keywords = {Human-AI interaction, Large language models, Personas, Text feedback, Writing assistance},
location = {Honolulu, HI, USA},
series = {CHI '24}
}