In Conversation: How Large Language Models Convincingly Mimic Human Conversation

What are the consequences of the discrepancy between reality and, as you call it in the paper, "media-mediated reality"?

Kückelhaus: In my view, the anthropomorphization of LLMs certainly comes with risks, because we don't constantly reflect on it. And that can have consequences for the user.

… such as the loss of critical distance?

Felder: This is where I disagree. Ultimately, we want our students, too, to reach a point where they judge statements on their own merits, not primarily by who made them. Perhaps generative AI can help put the emphasis back on the content. Of course, in everyday life we need reference points, for instance expert opinions we can rely on. At the moment, we're losing some of that sense of orientation. So it may actually be a good thing if dealing with LLMs prompts us to examine the claims being made more independently of the person when scrutinizing them for their truth and validity.

Kückelhaus: I can only agree with that to a limited extent. When information is linked to a particular individual, we can check it against reality and verify the facts. We can look into whether the person who made the statement is qualified in any way. With ChatGPT and other LLMs, that isn't possible, because we don't know what sources they are drawing on - even if a statement sounds completely convincing and the reasoning is perfectly coherent. Our tendency to rely too heavily on what we read or hear is a general problem, which becomes particularly acute in the context of language models. A great deal of trust - too much, in my view - is already being placed in this technology.

Felder: There are undoubtedly cases of improper use, and AI, of course, is not free of mistakes. That said, I find it problematic to cite this as a reason not to grapple with this new technology in the first place.

Kückelhaus: What we can both certainly agree on is the need for the thoughtful use of this technology. And that, in turn, also applies to language. Given the linguistic patterns that make the "machine" seem so human, we need to handle this thoughtfully and keep reminding ourselves that we are not dealing with another human.