# Sophistry on Steroids? The Ethics, Epistemology and Politics of Persuasive AI  ## Metadata - Author: [[Springer Vieweg]] - Full Title: Sophistry on Steroids? The Ethics, Epistemology and Politics of Persuasive AI - Category: #articles - Summary: insert summary - My notes: - Summary: The paper argues we should worry about persuasive AI for ethical, epistemic, and political reasons. The central worry is that a few powerful actors could flood and control the "marketplace of arguments" by producing persuasive content at scale. This may distort public debate, reinforce inequalities, and lead to less rationally formed attitudes. - URL: https://link.springer.com/article/10.1007/s00146-025-02624-z ## LLM Chats ## NotebookLM ## LLM Audio ## Highlights > To make this argument work, though, some reasons need to be given for thinking that lacking any skin in the game is a bad thing. Indeed, you might think that lacking any skin in the game is sometimes a *good* thing. It is precisely because human persuaders have skin in the game that they sometimes engage in lies, deceit and other forms of subterfuge to bring it about that other people believe things that will serve their (the persuaders’) interests. Of course, the purveyors of persuasive AI technologies can also engage in lies, deceit and subterfuge, so this might not be a point in favour of persuasive AI. But, equally, it doesn’t seem to be a point against it. ([View Highlight](https://read.readwise.io/read/01k977w98kmych1hbkwqv5n24f)) > Let me draw some general points from all this. First, there is good evidence that persuasive AI technologies (typically, LLMs) are at least as persuasive as humans, and there is some evidence that the most cutting-edge models are more persuasive than humans when they are given relevant information about the target audience. That providing relevant data about the target audience makes AI technologies more persuasive is not surprising, given that successful persuasive strategies and messages are typically tailored to their audience (Cialdini [2001](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR11); Maio et al. [2019](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR37); O’Keefe [2016](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR50)). LLMs are likely very good at tailoring due to their extensive training on large corpora that include numerous persuasive arguments, and those who have speculated about the persuasive power of AI have linked this to their ability to tailor their message for a target audience (Burtell and Woodside [2023](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR7); Floridi [2024](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR19)). ([View Highlight](https://read.readwise.io/read/01k99q2rdcnxphbv726dhaa9mq)) > What about other external influences? There are external influences that are not coercive or manipulative at all: people sometimes provide information with the simple intent to inform. Then there are external influences that are not overtly coercive or manipulative, but which may be coercive or manipulative in a more subtle way. Persuasion is a mode of influence that is meant to differ from, say, indoctrination precisely in that it is not overtly coercive or manipulative. But suspicion of persuasion, especially emotive and heavily rhetorical forms of it, has a venerable history in philosophy, stretching back to Plato (Dow [2019](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR15); McCoy [2007](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR42)). ([View Highlight](https://read.readwise.io/read/01k99q2ryvxc7r2epy8bdatxnh)) > The problem though is that, on a very wide range of issues—including many where persuasive AI technologies are likely to be deployed—there will be significant disagreements about whether these conditions are met, whether by particular persuasive AI technologies, or by particular uses of those technologies. For example, there is likely to be significant disagreement about whether persuasive AI technologies are being used to persuade people of things that are true because there is likely to be disagreement about what things are true. To be sure, there won’t *always* be disagreement about what things are true; consensus is sometimes arrived at in even the most heated of disputes. It may also be that, sometimes, disagreement is intractable because one side is simply being unreasonable. But the point is just that there will *often* be disagreement about what things are true, not that there will always be disagreement, or that any disagreement there might be is reasonable. As a result, the conditions on the permissible use of persuasive AI technologies are unlikely to resolve *political* problems that might result from the widespread deployment of persuasive AI technologies. Indeed, they are likely to create new problems, or at least new arenas within which existing political problems are debated and existing political battles fought. ([View Highlight](https://read.readwise.io/read/01k99yjp649mmkf57fd672wbqn)) > Some have worried that even rational forms of persuasion—persuasion by means of argument and reasons—can be coercive. In interpersonal contexts, you might worry that you cannot simply separate the persuasive power of an argument from the persuasive power of the person giving it, especially in contexts with power imbalances (Davis [2017](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR14); Tsai [2014](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR64)). ([View Highlight](https://read.readwise.io/read/01k99q9py4cgg80dy55cj7hbne)) > Persuasion is inherently challenging because humans are, by and large, *epistemically vigilant* (Mercier [2020](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR45); Mercier and Sperber [2017](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR46); Sperber et al. [2010](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR61)). One of the reasons why mass persuasion techniques such as marketing and propaganda typically have small (if any) impacts on people’s attitudes is that, for basic evolutionary reasons, we are primed to look for signs that someone is not trustworthy or does not have our best interests at heart. To be sure, humans can be tricked; we are not infallible. But we are far from the gullible creatures that many who are worried about the persuasive power of new technologies depict us as being (Mercier [2017](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR44)). It is therefore far from obvious that AI persuasion will ever be significantly more effective than human persuasion. There may simply be limits on how *persuadable* humans are. ([View Highlight](https://read.readwise.io/read/01k977s3wh4mgz9kd0661v8am7)) > The basic idea is simple enough: what is distinctive about rational modes of persuasion is that they involve providing *reasons* for adopting a new attitude or changing one’s existing attitudes. In their paper Mitchell and Douglas specify that by “reason” they mean a fact that counts in favour of (or against) the attitude in question ((Durmus [2024](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR16)), 5). For example, the fact that it will reduce rates of transmission is a reason to support a vaccination programme, whereas the fact that it will have serious side-effects for many who take it is a reason not to support it, or at least to only support a limited rollout of it. While I will follow Mitchell and Douglas in assuming that reasons are facts, someone who endorses a different view, e.g. a view on which reasons are mental states as in Turri ([2009](https://link.springer.com/article/10.1007/s00146-025-02624-z#ref-CR65)), can simply replace my view with their favoured view where relevant. ([View Highlight](https://read.readwise.io/read/01k977m54nmtcbpvdbyz4vb9ys)) > The first is that persuasive AI technologies have the potential to dominate the marketplace of arguments. For humans, contributing to the marketplace of arguments is time-consuming and labour-intensive—it involves coming up with new arguments, or finding ways of recycling old ones, and then looking for a way of getting those arguments disseminated more widely (e.g. publishing them somewhere where people will see them). Persuasive AI technologies provide a way of avoiding these time and labour constraints. The purveyors of persuasive AI technologies can generate huge numbers of “arguers for hire” (AI agents) or huge amounts of “persuasive content” within minutes, with any dilution in the quality of arguments often more than made up for by the massive increase in quantity. The second reason is that not just anyone can create and run persuasive AI technologies. The resources—not just financial, but also in terms of infrastructure, skills, and knowledge—required to design and operate a LLM are massive, so much so that only a very small number of companies operate in this field, at least at the level relevant to the argument of this paper (e.g. Anthropic, OpenAI, DeepMind, Microsoft, Meta). This means that the barriers to entering the marketplace in this way are very high, so high that ordinary people cannot hope to meet them. ([View Highlight](https://read.readwise.io/read/01k9a6gjyb2tx3q645bd2x2nsp)) > But, if it is a form of sophistry, it’s *sophistry on steroids*. The very small number of companies that operate these persuasive AI technologies have the power to create armies of “arguers for hire” at will, and so to distort the marketplace of arguments, shaping it towards their ends. ([View Highlight](https://read.readwise.io/read/01k9788y02yysrrt45mmvhj2ae)) > The marketplace of rationalizations. ([View Highlight](https://read.readwise.io/read/01k99yk23t1n8q1x07v47j5z3j))