Choice Over Control:
How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Abstract:
We propose a conceptual perspective on prompts for Large Language Models (LLMs) that distinguishes between (1) diegetic prompts (part of the narrative, e.g. "Once upon a time, I saw a fox..."), and (2) non-diegetic prompts (external, e.g. "Write about the adventures of the fox."). With this lens, we study how 129 crowd workers on Prolific write short texts with different user interfaces (1 vs 3 suggestions, with/out non-diegetic prompts; implemented with GPT-3): When the interface offered multiple suggestions and provided an option for non-diegetic prompting, participants preferred choosing from multiple suggestions over controlling them via non-diegetic prompts. When participants provided non-diegetic prompts it was to ask for inspiration, topics or facts. Single suggestions in particular were guided both with diegetic and non-diegetic information. This work informs human-AI interaction with generative models by revealing that (1) writing non-diegetic prompts requires effort, (2) people combine diegetic and non-diegetic prompting, and (3) they use their draft (i.e. diegetic information) and suggestion timing to strategically guide LLMs.
Overview of our four UI variants, showing the user’s written text (black font, i.e. a diegetic prompt), the suggestions (text highlighted in green, and options in the list), and a popup text box that allows users to input an instruction as a zero-shot prompt to the system (i.e. a non-diegetic prompt).
BibTeX:
@inproceedings{10.1145/3544548.3580969,
author = {Dang, Hai and Goller, Sven and Lehmann, Florian and Buschek, Daniel},
title = {Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting},
year = {2023},
isbn = {9781450394215},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3544548.3580969},
doi = {10.1145/3544548.3580969},
abstract = {We propose a conceptual perspective on prompts for Large Language Models (LLMs) that distinguishes between (1) diegetic prompts (part of the narrative, e.g. “Once upon a time, I saw a fox...”), and (2) non-diegetic prompts (external, e.g. “Write about the adventures of the fox.”). With this lens, we study how 129 crowd workers on Prolific write short texts with different user interfaces (1 vs 3 suggestions, with/out non-diegetic prompts; implemented with GPT-3): When the interface offered multiple suggestions and provided an option for non-diegetic prompting, participants preferred choosing from multiple suggestions over controlling them via non-diegetic prompts. When participants provided non-diegetic prompts it was to ask for inspiration, topics or facts. Single suggestions in particular were guided both with diegetic and non-diegetic information. This work informs human-AI interaction with generative models by revealing that (1) writing non-diegetic prompts requires effort, (2) people combine diegetic and non-diegetic prompting, and (3) they use their draft (i.e. diegetic information) and suggestion timing to strategically guide LLMs.},
booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems},
articleno = {408},
numpages = {17},
keywords = {Co-creative systems, Human-AI collaboration, Large language models, User-centric natural language generation},
location = {Hamburg, Germany},
series = {CHI '23}
}