The Environmental Aesthetics of Generative AI submitted to be cut down by a few thousand words 8 May 2026.md File Below is a chat with Opus about cutting a paper of mine. In the course of the conversation you'll see I decided I want to cut it down to 8,000 words. Okay. And I've attached the the paper in question. What I would like you to do is evaluate Claude's suggestions about how to achieve 8,000 words. And give me your view on what I should do. It can be as similar or as different to Opus as you like. CHAT: \*\*You\*\* This text is about 12,000 words (I think). How could we cut it down to 10,000, 9,000, 8000? Give us various options. Your brief with each suggestion is the text has to be as faithful to the original as possible, but in each an every suggestion you can make decisions about what counts as the core of the text and what is cutable, or whether it should be thinned out, or cut, or a mixture of the two. and little cuts or whole ideas etc.? you get the idea? obviously make sure that you explain your analysis, justification, reasons etc etc. for your answer BEFORE giving me your final answer. Please give me a long, deep, and detailed answer, with lots of options/ideas: as we are in the 'kicking ideas around' stage, I want as much feedback as you can give me so that i have a lot to consider. --- \*\*Claude\*\* Right — let me first establish what we're actually working with, then sketch out the editorial logic, and then give you concrete options at each target. ## Where the words actually are The body comes to \*\*13,166 words\*\* (excluding the references, which add another 770). So you're not at 12,000 — you're at roughly 13,000–13,100 of "real" prose. That changes the maths slightly: cutting to 10,000 is ~24%, to 9,000 is ~32%, to 8,000 is ~39%. Fairly aggressive at the lower end. Section-by-section breakdown of the body: | Section | Words | % | | | --- | --- | --- | --- | | Introduction | 363 | 2.8% | tight already | | §1. Design vs. Order | 1,405 | 10.7% | dense scaffolding | | §2. What LLMs Are | 1,468 | 11.1% | technical primer | | §3 intro | 266 | 2.0% | | | §3.1 Make-believe | 985 | 7.5% | Mallory + Cross | | §3.2 Concessive mindedness | 814 | 6.2% | Frankish | | §3.3 Post-training, chat personae | 728 | 5.5% | objection-handling | | §4. Artifacts | 1,739 | 13.2% | heaviest single section | | §5.1 Textual Regularities | 1,777 | 13.5% | tied for heaviest | | §5.2 Practical Acquaintance | 657 | 5.0% | | | §6 intro | 170 | 1.3% | | | §6.1 Outputs | 1,079 | 8.2% | two case studies | | §6.2 Chats | 517 | 3.9% | tight | | §6.3 Models | 906 | 6.9% | several digressions | | Conclusion | 212 | 1.6% | tight | §4 and §5.1 together are 3,516 words — over a quarter of the paper. §3 in toto is 2,793 — over a fifth. These are your big targets. ## Diagnostic: what's load-bearing and what's not Before suggesting cuts, here's how I'd map the argument's spine against the prose: \*\*The six-move spine of the paper:\*\* 1. Carlson distinguishes design from order appreciation (§1) 2. LLMs are token-predictors trained on text; their organisation emerges from training (§2) 3. Person-based aesthetics is wrong for LLMs (§3) 4. Pure design aesthetics is wrong for LLMs (§4) 5. Order appreciation, guided by \*semiotic physics\*, is right (§5) 6. It applies at three levels: outputs, chats, models (§6) Every section earns its keep in principle, but several sections do their work two or three times over, and a few do work that the spine doesn't strictly need. \*\*Most expandable (highest cut-yield-to-damage ratio):\*\* - \*\*§3.3 (728 words).\*\* This reads as a recently-added objection-handler. The substantive claim — that post-training chat personae are recurring response profiles, not subjects with lives — is already implied by §3.1 and §3.2 plus the post-training apparatus introduced in §2. The section adds rhetorical reinforcement and the Farrell/Gopnik/Shalizi/Evans quote, but the spine survives without it. Cuttable in whole or substantially trimmed. - \*\*§4 (1,739 words).\*\* Has noticeable redundancy. The "form follows function" point is rehearsed three times; the move from Carlson to Forsey to Parsons-and-Carlson is set up at length but they all converge on the same claim; the raku/Pollock analogies pull in the same direction (one would do); the "design has its place but the action is in emergence" upshot is restated in three different paragraphs. Probably 400–600 words of slack here without touching anything load-bearing. - \*\*§5.1 (1,777 words).\*\* Has a similar shape. The mechanistic-interpretability detour (line 173) is useful but expensively staged via the chemistry/geology analogy; Janus, Picca, and Wolfram are each quoted when one extended treatment plus a note on the others would suffice; the Pollock parallel reappears here when it could live just in §4; the aspection-shift paragraph (line 189) and the language-as-object paragraph (line 191) restate the shift in two passes. - \*\*§6.1's two case studies.\*\* The reasoning case and the bee text both illustrate the same thesis (semiotic physics makes order visible). Strategically you might keep just the bee text — it's the more striking and revealing illustration, and Carlson's "some cases reveal order better than others" remark gives you cover for asymmetric treatment. - \*\*§6.3's diversions.\*\* The video-game analogy (~150 words), the "distinguishing appreciation from capability/safety/benchmarking" paragraph (~120 words), and the "mirror of culture" future-work flag (~120 words) are all detachable. None is on the spine. - \*\*Quotation overhead.\*\* The long Carlson quote on order appreciation (line 33), the form-follows-function quote (line 27), the Olah "growing not building" quote (line 71), the Wolfram trajectory-in-feature-space quote (line 177), and the Carlson-via-Janson Pollock quote (line 187) — most could be paraphrased at half the length without loss. \*\*Most load-bearing (touch with care):\*\* - The §1 Carlson framework. You need design vs. order on the table. But §1.x on person appreciation (lines 39–43) is doing optional work — the three options for extending Carlson to persons could collapse to a single sentence, since you don't need to settle which is right. - §2's technical content from "embeddings" through "post-training" — the philosophy depends on it. - §3.2 (Frankish/concessive mindedness). This argument is distinct from the make-believe argument and is the more philosophically substantive of the two negative subsections. - §5.1's central claim (semiotic physics articulates causal forces at the level of perceivable text), §5.2 (practical acquaintance), §6.2 (chats as environments), §6.3 vibe-paragraph. ## Three editorial philosophies You can think of cuts as falling under three increasingly aggressive approaches. The right mix probably uses Approach A throughout and reaches for B and C as the target shrinks. \*\*Approach A — Distributed thinning.\*\* Tighten prose everywhere; paraphrase quotes; remove signposting and recaps; collapse paired examples; reduce hedges. Preserves every argument and section. Realistic yield: 1,200–1,800 words across the paper. Gets you comfortably to 11,500 and within reach of 11,000. Risk: low. Cost: some of the pedagogical breathing space goes; the paper feels denser. \*\*Approach B — Targeted excision.\*\* Identify discrete subsections or whole paragraphs that can be cut without rewriting the spine. Candidates above. Yield: another 1,000–1,500 words on top of A. Gets you to 10,000–10,500. Risk: moderate (some readers may want what's cut). Cost: certain reinforcing moves and counter-objections disappear. \*\*Approach C — Structural compression.\*\* Merge sections, drop one of two parallel illustrations, reframe the technical primer in terms appropriate to a sophisticated audience rather than as a tutorial, fold the negative arguments into a single tighter §3. Yield: another 1,000–1,500 words on top of A+B. Gets you to 8,000–9,000. Risk: higher (rewriting required, and the paper's leisurely diagnostic feel is lost). Cost: some readers might find the negative argument too compressed; the tutorial-friendly §2 becomes opaque to non-specialists. ## Concrete cut packages I'll give you several options at each target, varying which approach dominates. ### Target: ~10,000 words (~3,100 words to cut) #### Option 10A — "Conservative thinning" Pure Approach A. Walk through every section trimming 15–25% from each by: - paraphrasing the Carlson order-appreciation quote (line 33) and the form-follows-function quote (line 27); - shortening the cat-on-the-mat worked example in §2; - paraphrasing the Olah quote; - trimming the §1 person-appreciation discussion to a single paragraph saying "if Carlson's framework extends to persons at all, it must do so under his recommendation"; - collapsing the cliff-face / Rembrandt double example in §1 to one; - removing recap clauses ("As established in §2…", "Having set aside…"); - paraphrasing all block quotes over 40 words. What survives: every argument, every example, every named interlocutor. What goes: pedagogical generosity, some breathing room. \*\*Faithfulness: very high.\*\* Best if you want a version someone reading both could see as "the same paper, polished." #### Option 10B — "Light surgery + thinning" Approach A across the board, plus: - \*\*Cut §3.3 entirely\*\* (–728 words). Move one or two sentences about post-training and chat personae into §3.2 and into §2's closing paragraph. The Farrell/Gopnik/Shalizi/Evans quote can move to a footnote in §3 conclusion or §4 opening. - \*\*Trim §4 by ~300 words\*\*, mainly by dropping the explicit setup of Forsey and Parsons-Carlson as separate views (mention them more in passing) and choosing either raku \*or\* Pollock as your hybrid analogy. - \*\*Trim §5.1 by ~300 words\*\*, mainly by paraphrasing the Wolfram quote and folding the "three objects to investigate" gloss into a single sentence; trimming the language-as-object paragraph (line 191), which restates the aspection-shift point of line 189. - Light thinning elsewhere. What goes: §3.3's redundant reinforcement; some of §4's catalogue of design-aesthetic theorists; one of two craft-and-art analogies. What survives: the spine intact; both case studies in §6.1; both negative arguments at full strength; the practical-acquaintance discussion. \*\*Faithfulness: high.\*\* Probably my recommendation if you can spare losing §3.3 (which I'd argue you can). #### Option 10C — "Keep §3.3, surgery elsewhere" For the case where you want to keep §3.3 because it's responding to a specific reviewer or audience concern: - Trim §3.3 by ~250 words (cut the lengthy setup and the closing paragraph; keep the assistant-as-pattern-not-mechanism core). - Cut §6.3's video-game analogy (–150 words) and "mirror of culture" paragraph (–120 words). - Cut or compress the §6.3 "distinguishing from capability/safety/benchmarking" paragraph (–100 words). - Compress §6.1's reasoning case study by ~250 words, keeping the bee text at full length. - Approach A thinning elsewhere. \*\*Faithfulness: high.\*\* Good if §3.3 is non-negotiable. ### Target: ~9,000 words (~4,100 words to cut) At this target you have to choose at least one of: - Sacrificing §3.3 entirely; - Sacrificing one of the two case studies in §6.1; - Substantially compressing §2's pedagogical primer; - Removing §1's person-appreciation discussion (and rebuilding the link to §3 in a sentence); - Removing §6.3's diversions wholesale. #### Option 9A — "Cut the tertiary moves" - Cut §3.3 entirely (–728). - Compress the reasoning case study in §6.1 to a brief paragraph (–250 to –300); keep bee text full. - Cut §6.3's video-game analogy, future-work paragraph on culture, and the capability/safety/benchmarking distinction paragraph (–350 to –400). - Trim §1's person-appreciation digression to one sentence (–250). - Approach A thinning across §2, §4, §5.1 to yield another 1,500–1,800. What survives: spine, both negative arguments at strength, full bee-text discussion, model-level argument intact. What goes: redundant objection-handling, one illustrative pair, three diversions, a side-discussion about persons. \*\*Faithfulness: moderate-to-high.\*\* Probably the cleanest 9K option. #### Option 9B — "Compress the technical primer" - Halve §2 from ~1,470 to ~700 words (–770). For a philosophy of perception/mind audience, the cat-on-the-mat tutorial is overcatered. You can do "tokens, embeddings, autoregressive sampling, RLHF" in 700 words for this audience without losing what the philosophy needs. - Cut §3.3 (–728). - Trim §4 by ~400. - Trim §5.1 by ~400. - Light Approach A elsewhere. What goes: pedagogical accessibility for non-specialists; some §4/§5.1 throat-clearing. \*\*Faithfulness: moderate.\*\* This is the right option if your target venue's readership is technically sophisticated. Risk: if reviewers come from aesthetics-only backgrounds, the compressed §2 may feel jumpy. #### Option 9C — "Merge §3.1 + §3.2 + §3.3 into one tighter §3" - Rewrite §3 as a single argument with three sub-moves rather than three subsections (–500 to –700 vs. current §3.1+§3.2+§3.3 of 2,527 words). The Cross discussion can compress to a footnote; Mallory's chatbot fictionalism gets a tight summary; Frankish gets the slightly longer treatment because his argument is the more demanding; the §3.3 material becomes a closing two-paragraph caveat. - Trim §4 by ~300 and §5.1 by ~300. - Cut §6.3 diversions (–350). - Approach A elsewhere. \*\*Faithfulness: moderate.\*\* This option has the merit of producing a tighter, more readable §3, which is currently a little baggy. It's a deeper rewrite though. ### Target: ~8,000 words (~5,100 words to cut) This is heavy and requires structural moves. Here are three ways to land it. #### Option 8A — "Maximal targeted excision" - Cut §3.3 entirely (–728). - Halve §2 (–770). - Halve one of the §6.1 case studies (–400). - Cut §1's person-appreciation discussion (–250) — replace with one sentence. - Cut §6.3's three diversions (–400). - Trim §4 aggressively to ~1,100 (–600). - Trim §5.1 aggressively to ~1,200 (–600). - Trim §3.1's Cross discussion (–250); keep the Mallory analysis lean. - Approach A thinning elsewhere (~–200). This is brutal but additive: you keep every section but trim each substantially. The paper would feel notably more clipped, but every argument would still be there. #### Option 8B — "Restructure §3 + §4 into a single negative section" - Combine §3 and §4 into one negative section ("Why person-based and design-based appreciation are both wrong") of ~1,800 words instead of the current 4,532. Yield: ~–2,700. - Trim §2 by 400, §5.1 by 400, §6 by 500. - Approach A elsewhere. This reduces the paper's "diagnostic" feel — currently you walk the reader carefully through \*why\* each tempting option fails before turning to your positive view. The merged structure is more efficient but loses the rhetorical patience. #### Option 8C — "Drop one case study; absorb the Pollock/raku analogy into §5" - Cut the reasoning case study from §6.1 (~–400). Bee text alone. Carlson's "some cases reveal better than others" remark covers this. - Cut the raku/Pollock analogies from §4 and develop the Pollock parallel only in §5.1 where it does its real work (–300 net from the move; –250 in §4). - Cut §3.3 (–728). - Halve §2 (–770). - Cut §1's person-appreciation discussion (–250). - Cut §6.3 diversions (–400). - Trim §4 by another ~300 and §5.1 by ~400. - Approach A elsewhere. Probably the most coherent 8K version. The Pollock parallel benefits from being made once, properly, in §5 where it's doing its main work. The reasoning case is probably the more dispensable of the two §6.1 illustrations. ## Cross-cutting prose-level moves you can apply at any target Independent of which package you adopt: - \*\*Quote audit.\*\* Every block quote over 40 words should justify itself. The Carlson order-appreciation block (line 33) is doing real work but could be 40 words instead of 80. The Olah "we grow them" passage is rhetorically nice but the substance is one sentence. The Wolfram "linguistic feature space" quote can become a paraphrase that takes one line. Net yield from this alone is probably 250–400 words. - \*\*Recap clause cull.\*\* Many paragraphs open with "As we established in §2…" or "Having set aside the person-based options…" or "Section 2 already noted that…". These are individually small but collectively account for 200+ words. Most can go. - \*\*Hedging compression.\*\* "It is therefore not surprising that…", "In a sense, of course…", "What we might call…", "It will be enough to note that…". A pass on these alone yields 100–200 words. - \*\*Paired-example collapse.\*\* Most of your examples come in pairs (cliff-face and Rembrandt; chair and bridge; raku and Pollock; reasoning text and bee text; farmer and gardener and forester). One of each pair is usually doing the work. Pick the more vivid and cut the redundant partner. - \*\*Section-opener compression.\*\* Every section currently opens with a paragraph that recaps where we are and announces what comes next. Most could be a sentence. - \*\*The "we shall see" / "as we will argue" gestures.\*\* These are forward-looking promises that the prose doesn't really need; the reader will see when they get there. ## What I'd recommend If you asked me to pick: \*\*Option 10B\*\* is the cleanest 10K landing. \*\*Option 9A\*\* is the cleanest 9K. \*\*Option 8C\*\* is the most defensible 8K. The single highest-yield move is cutting §3.3 — I genuinely think the spine doesn't need it, and what it adds (the chat-persona-as-pattern point, the Farrell/Gopnik/Shalizi/Evans quote) can be redistributed in two or three sentences. After that, the highest-yield moves are paragraph-by-paragraph thinning of §4 and §5.1, and dropping the reasoning case study from §6.1 if you can bear it. The lowest-yield-but-highest-pain moves would be touching §3.2 (Frankish), §5.2 (practical acquaintance), §6.2 (chats), or the conclusion. These are all already lean and load-bearing; cutting here would buy little and cost much. A general sanity check: the paper as it stands has the architecture of a 12–14K paper, and you can get it to 10K without rebuilding anything. Below 10K you're choosing what to lose, not just compressing what you have. So I'd think hard about whether 10K is acceptable for the venue — a lot of the philosophical generosity of this draft is in the diagnostic patience of §3 and §4, and you start to lose that around 9K. Happy to drill into any specific package, or to actually do a sample tightening pass on one section so you can see what Approach A looks like in practice. --- \*\*You\*\* I want to work on getting it chopped down to 8,000 words. Okay, however when you gave me those options of 8A, 8B, 8C, etc. they weren't written clearly enough. me to understand what you were suggesting. --- \*\*Claude\*\* I see the user found my previous explanations of the 8K options too vague—I need to be more concrete about what each version actually looks like and how they differ in substance, not just word counts. Let me reconsider what made those explanations fall short and focus on showing the actual shape and character of each option rather than just listing cuts. I need to be clearer about what each option actually produces. For 8A, I'm keeping the full six-move structure but trimming everywhere — tightening quotes, removing redundancies, compressing examples, and dropping §3.3 entirely. The paper stays the same shape but feels denser and loses some breathing room. For 8B, I'm collapsing the two negative sections (§3 and §4) into a single argument that both person-aesthetics and pure design-aesthetics fail, which shifts weight toward the positive case for semiotic physics but sacrifices the diagnostic patience the current structure provides. For 8C, I'm keeping the original structure but cutting the reasoning case study in §6.1 and keeping only the bee example, removing the redundant Pollock parallel from §4, and trimming various digressions throughout. This preserves the more memorable illustration while losing some analogical depth. I should probably consider a few other directions too — maybe something that focuses the argument even more tightly, or explores whether there's a way to strengthen the positive case without cutting so much. One angle: what if I drop the design-aesthetics critique in §4 entirely and fold its core claim into §5 instead? That would let §5 do the real argumentative work as the contrast, though it's a bit risky since §4 sets up something important. Another direction: what if I just compress the entire setup — §1, §2, and §3 — down to their essentials and save the full strength for the positive argument where it matters most? I'm settling on a clearer structure for presenting these options. Each one needs a straightforward explanation of what gets removed, how the paper reads afterward, and what trade-offs come with it. I'll use focused paragraphs with headers rather than bullet points to match the plain-language style, and keep the language concrete and direct so the choice becomes obvious. For Option 8A specifically, I'm cutting §3.3 entirely and folding its content into nearby sections, trimming the personal digression in §1 down to a single sentence, removing the video-game analogy and related tangents from §6.3, and then systematically reducing the longer sections—halving §2 by dropping the cat-on-the-mat example and streamlining the technical narrative, cutting §4 by keeping only the Pollock analogy and trimming the catalogue material, and condensing §5.1 by paraphrasing the Wolfram quote and consolidating the language-related discussion. The result is the same six-move argument with all section titles and core claims intact, but the prose is noticeably denser and less pedagogical—§2 now reads more like a primer for sophisticated readers, §3 has two subsections instead of three, and §5.1 has fewer interlocutors. The overall reading experience is clipped and demanding rather than generous, with block quotes over 40 words paraphrased and paired examples compressed throughout. What I'm losing is the leisurely diagnostic tone and the extended Cross discussion that thins to a single paragraph; the Wolfram quote becomes a paraphrase rather than a full citation. Now I'm considering Option 8B: merging the two negative sections. Instead of spending 4,532 words across §3 and §4 arguing separately that person-aesthetics and design-aesthetics both fail, I'd combine them into one ~2,000-word section. That saves roughly 2,500 words right there, plus another 400 from trimming §6.3 diversions. The structure would compress to five moves: Carlson's framework, what LLMs actually are, why both rival aesthetics collapse, semiotic physics as the solution, then the three-level model. The merged negative section would lay out the temptations first, then spend about 700 words dismantling the person-based view (Mallory and Frankish in tighter form), another 700 on pure design-aesthetics, and close with a 200-word bridge to the positive argument. The trade-off is losing the patient three-subsection treatment of person-based options—Mallory gets one paragraph instead of several, Frankish gets two instead of a full subsection, and the chat-based counterargument gets compressed significantly. The reading experience shifts: the negative case moves faster while the positive case gains proportional weight, creating a clearer "here's what fails, here's what works" trajectory. Reviewers expecting careful step-by-step engagement with each rival might find the negative section rushed, but since that case is mostly diagnostic and depends on accepting Carlson's framework, the core argument doesn't require extended treatment. Now I'm mapping out a more aggressive consolidation—dropping the reasoning-style case study entirely, keeping only raku in the aesthetics section while moving Pollock later where it carries more weight, cutting the person-appreciation digression and most of §3.3, halving §2 down to around 700 words, and trimming the diversions in §6.3 about video games and capability concerns. This saves substantial space while keeping the spine intact. So I'm looking at the trade-offs in this 8C version — dropping one of the two case studies in §6.1 means I lose the chance to show that semiotic physics works across both ordinary and unusual outputs. The bee text alone demonstrates that order emerges where it seems absent, but the reasoning case was doing different work, showing the principle applies more broadly. That's a real loss, though the version stays philosophically coherent and is straightforward to execute since it's mostly trimming rather than restructuring. Now I'm considering 8D as a separate option — compressing the setup sections (§1, §2, §3) down to bare essentials in plain prose, then giving the later sections (§4, §5, §6) their full development. This would flip the proportions: thin introduction, thick philosophical and analytical core. the positive proposal feels immediate and direct. The compressed diagnostic might strike some readers as thin, but others will appreciate cutting straight to the substantive argument. I'm thinking I should present the user with several distinct options rather than just one path forward—maybe four approaches with genuinely different philosophies, then flag which one I'd lean toward recommending. The real tension I'm seeing is between 8A and 8C: one keeps both case studies but aggressively trims the setup, while the other sacrifices a case study entirely to preserve the philosophical depth of the middle sections. I should also explore whether there's a way to protect §3 and §4's full argument by cutting more strategically elsewhere. Option 8E is emerging as a possibility: keep the diagnostic core of §3 and §4 intact, but cut hard from the technical primer in §2, drop one of the case studies, trim the model-level discussion in §6.3, and remove the person-appreciation tangent from §1. This would mean reducing §2 to around 600 words, excising the reasoning case study, lightening the bee-text section, compressing §6.3 to focus only on the vibe paragraph while dropping the video-game and capability discussions, cutting the §1 digression, removing §3.3 entirely, and doing light prose tightening across §3, §4, and §5. The resulting paper would have the negative arguments fully present and the positive case intact, but with a bare-bones technical primer, only the bee text in §6.1, and §6.3 as a single argumentative thread without diversions. This works well for readers already familiar with LLMs, though it risks being too thin for a mixed audience unfamiliar with the basics. Now I'm settling on which option to actually present. Given the user's earlier frustration with unclear options, I should narrow this down to 3-4 clear choices, make explicit what each version looks like, and use a consistent template so they're easy to compare. I'm going with four distinct philosophies: an even trim across the board, a structural merge of the negative sections, dropping one case study entirely, and front-loading a thin version. Let me reframe these more clearly—the first approach trims everything proportionally without cutting any subsections entirely, keeping all the moves and examples but tighter overall. The second combines §3 and §4 into a single negative section... Now I need to map out what each version would actually contain, and I should also factor in the user's stylistic preferences—British English, avoiding value-laden language like "important" or "crucial"... The user's style guidance is pulling in a different direction from the earlier preferences. They want plain language, direct and affectless, no academic flourishes—straightforward vocabulary that explains complex ideas without unnecessary latinate constructions or interpretive language. That's more recent and explicit, so it takes priority. Now I'm drafting the response. I want to be concrete about each option, but I also need to respect length. The user asked for clarity, not brevity, so I'll give them the detail they need to understand each path forward, just in plain language. For each option, I'll lay out the editorial idea, show exactly what gets cut and where, and describe what the paper looks like after. I'm thinking a table might work for comparison, but I should write it out first and see if that's clearer than prose. --- \*\*You\*\* "If you want a paper that reads more directly and has a sharper argumentative arc, Option 2 is the better choice. The merged negative section is in some ways more rhetorically effective than the current two separate sections, even though it loses some material."i think something option b flavoured is the best. But I also think that Section 2 could be much improved, and perhaps that this will lead to improved sections 5 and 6. --- \*\*Claude\*\* Good. Option 2 is the right structural move, and you've spotted something real about §2. ## What §2 is currently doing wrong §2 is a tutorial. It walks the reader through tokens, then probabilities, then pre-training, then embeddings, then attention, then autoregressive decoding (with the cat example a second time), then post-training, then product wrapping. By the end the reader understands how an LLM works. That's the wrong design brief. §2 needs to put on the table the technical primitives that the rest of the paper will use. It is currently organised in pedagogical order ("here is what you need to know first, then next") rather than in argumentative order ("here is what §3 will reject, here is what §5 will redescribe, here is what §6 will redeploy at three scales"). The cost of this mismatch is most visible in §5.1. §5.1 takes 1,780 words partly because it has to re-articulate what embeddings, attention, and post-training actually produce in the text — vocabulary clustering, coherence dynamics, register stability, alignment markers. §2 introduced the mechanisms. §5.1 introduces the textual effects. These should be one continuous setup, not two. §6 has a similar problem at smaller scale. Its three levels (outputs, chats, models) are introduced as a fresh conceptual move, with the tree-forest-biosphere analogy doing the work of presenting them. But the three levels are already implicit in §2's technical material: an output is one generation; a chat is context accumulating across turns; the model is the trained system itself. §2 could put the three levels on the table directly, and §6 could lean on §2 rather than reintroducing them. ## What §2 should do instead Restructure §2 around four primitives, in the order the rest of the paper will use them: \*\*1\\. Token-level operation.\*\* The system generates text by repeatedly sampling next tokens from learned probability distributions. One paragraph. No worked example. \*\*2\\. The trained landscape.\*\* Pre-training produces three features that matter for later sections: embeddings (which produce semantic clustering), attention (which tracks long-range dependencies), and a probability distribution over continuations that reflects regularities in the training corpus. Two paragraphs. \*\*3\\. Post-training shapes the landscape.\*\* RLHF biases the system toward assistant-like patterns. Certain regions of the behavioural space become easy to reach; others become hard to reach. One paragraph. \*\*4\\. Grown, not designed.\*\* Designers specify the architecture and training objective; the organisation emerges from training. Olah's point. One paragraph. Then a closing paragraph that previews the three scales at which the trained system can be characterised: a generation as one trajectory through the landscape; a chat as path-dependent context accumulation across turns; the model as the underlying landscape that produces both. Target: about 700 words, down from 1,470. ## What this does for §5 If §2 has named embeddings as producers of semantic clustering, attention as the mechanism behind long-range dependencies, and post-training as the source of assistant-like patterns, then §5.1 does not need to reintroduce any of this. §5.1's job becomes: (i) These features have textual signatures the reader can perceive — vocabulary clustering, contextual threading, register stability, alignment markers. (ii) "Forces" is the right word for these factors: they are causes of how text propagates, not metaphorical projections. (iii) Knowledge of these forces guides aspection in the way geological knowledge guides aspection of a landscape. Three moves. About 1,000 words instead of 1,780. The chemistry/geology detour about mechanistic interpretability becomes much shorter, because the question "why work at the textual level rather than the circuit level?" has a direct answer once §2's primitives are already textual-effect concepts. The Janus, Picca, and Wolfram material compresses to one paragraph that names the literature as the source of the term "semiotic physics" rather than three blocks each developing a different framing. §5.2 (practical acquaintance) is mostly fine as it stands and only needs light tightening. ## What this does for §6 §6's introductory paragraph (170 words) and the tree-forest-biosphere analogy can become two sentences, because §2's closing paragraph has already put the three scales on the table. §6.3 ("Appreciating Models") currently spends time explaining what it means to appreciate the underlying model rather than its outputs. With §2 having named the model as the learned landscape, §6.3 moves straight to characterising what model-level appreciation involves. The video-game analogy and the capability/safety/benchmarking paragraph come out. The "mirror of culture" paragraph comes out (it is a flagged future-work gesture). §6.2 (chats) gains less directly but still benefits: path-dependence is already a feature named in §2, not a fresh introduction. §6.1's two case studies remain the question. With the §2 reorganisation, the reasoning case becomes more obviously redundant — the reader already has the conceptual primitives, and the bee text alone can carry the illustrative work. Cut the reasoning case. ## The merged negative section, with the new §2 in place Merging §3 and §4 gives you about 2,000 words for the negative argument. With §2 doing more conceptual work, this is workable. A shape that hangs together: \*\*Setup (150 words).\*\* §2 has described what LLMs are. We now ask whether the natural ways of approaching them aesthetically — as persons or as designed tools — fit what they are. \*\*Against person-based aesthetics (700 words).\*\* Three compressed sub-moves: make-believe (Mallory and Cross together, ~250 words); concessive mindedness (Frankish, ~350 words); chat personae as recurring patterns rather than subjects (the §3.3 point in ~100 words). The first and third move can be one paragraph each; Frankish gets two. \*\*Against pure design aesthetics (900 words).\*\* Take the design-aesthetic view seriously as a starting position; identify the gap (the order in an LLM is grown, not imposed); use raku alone as the hybrid-craft analogy and let Pollock travel to §5.1 where it does its real work on the "forces" claim. Forsey and Parsons-and-Carlson get one sentence each rather than full setup paragraphs. \*\*Transition (250 words).\*\* The two negative arguments converge: what matters aesthetically is the emergent linguistic order. §5 develops an aesthetics that takes that order as its primary object. ## Revised section budget | Section | Current | Target | | --- | --- | --- | | Introduction | 363 | 300 | | §1 (Carlson's framework) | 1,405 | 1,000 | | §2 (What LLMs Are) | 1,468 | 700 | | Merged negative section | 4,532 | 2,000 | | Semiotic physics (was §5.1 + §5.2) | 2,434 | 1,400 | | Three levels (was §6) | 2,672 | 1,600 | | Conclusion | 212 | 200 | | \*\*Total\*\* | \*\*~13,100\*\* | \*\*~7,200\*\* | That gives 800 words of headroom against the 8,000 target. You will probably need some of it back when individual paragraphs resist compression. The §1 cut is the one I'm least confident about; if it stays at 1,200 instead of 1,000, the total is 7,400, still under target. ## Order of work Do these in sequence: \*\*1\\. Rewrite §2 first.\*\* Don't compress the existing version paragraph by paragraph. Write a fresh §2 organised around the four primitives, with the closing paragraph that previews the three scales. Aim for 700 words. \*\*2\\. Draft the merged negative section.\*\* Once §2 is settled, the negative argument can be written as a continuous piece rather than as a stitched-together §3 and §4. The make-believe and concessive-mindedness compressions are the only parts that need careful surgery; the design-aesthetic material has obvious slack once it isn't framed as a standalone section. \*\*3\\. Trim §5 and §6 last.\*\* With §2 doing more work, both sections need adjustment anyway. Easier to do this after §2 is settled than speculatively now. The §2 rewrite is the move that unlocks the rest. It changes what §5 and §6 have to do, which makes the cuts in §5 and §6 easier to identify and justify. If you want, I can take a first pass at the rewritten §2 — keeping the technical content faithful to your current draft, but reorganised around the four primitives and with the new closing paragraph. That gives you something concrete to react to before you draft the merged negative section. --- ## The Environmental Aesthetics of Generative AI ## Introduction In recent years, aestheticians and philosophers of art have turned their attention towards generative AI — e.g. whether AI systems can be authors or co-authors, whether AI-generated work has any aesthetic merit at all (Wojtkiewicz 2023; Cross 2025). Carlson's aesthetics of natural environments, we argue, offers a productive approach to this territory and opens up the possibility that LLMs themselves can be appreciated. Two temptations should be resisted. The first is to appreciate LLMs as persons. Users talk about a model's 'personality' or 'vibe', and it is natural to respond aesthetically to these apparent traits. But LLMs lack the temporally extended life, the stable dispositions and projects, that underwrite person appreciation. The second is to treat LLMs simply as designed artifacts. LLMs are artifacts, but their aesthetically relevant features — the patterns in their outputs, their characteristic 'feel' — emerge from training rather than being specified by designers. Order appreciation offers an alternative. Carlson argues that we appreciate nature by attending to patterns produced by natural forces, guided by scientific knowledge — geology, ecology, and the like — that makes those patterns visible. LLMs call for something similar: attention to patterns produced by training, guided by what we call semiotic physics — knowledge of how mechanisms such as embeddings and reinforcement learning shape generated text. This framework applies at three levels: outputs as specimens, chats as environments, and models as the ground of order. The result is an aesthetics that treats LLMs neither as quasi-persons nor as ordinary tools, but as generative systems with their own characteristic dynamics. The paper proceeds as follows. Section 1 sets out Carlson's distinction between design appreciation and order appreciation, and considers how person appreciation might fit into this framework. Section 2 describes what LLMs are at a schematic level: token-based predictors trained on large text corpora and shaped by reinforcement learning. Sections 3 and 4 develop the negative arguments: §3 argues against appreciating LLMs as persons; §4 argues against simple design appreciation. Sections 5 and 6 develop the positive account: §5 introduces semiotic physics as the right kind of knowledge for order appreciation of LLMs; §6 shows how this framework guides appreciation at the three levels. --- ## 1\\. Appreciating Design, Appreciating Order Both our criticism of agentive views and our positive account will draw from Carlson’s environmental aesthetics, as laid out in his 2000 book \*Aesthetics and the Environment\*. We start with Carlson's general recommendation for aesthetic appreciation: take things as what they are, and look at them in the light of the right kind of knowledge. as in our appreciation of works of art, we must appreciate nature as what it in fact is, that is, as natural and as an environment. Second, it recommends that we must appreciate nature in light of our knowledge of what it is, that is, in light of knowledge provided by the natural sciences, especially the environmental sciences such as geology, biology, and ecology. (Carlson, 2000, p. 6) This captures something quite intuitive about how we appreciate nature versus how we appreciate works of art. Consider what goes wrong when we depart from it. If we accept, as a majority do in the 21st century, that mountains and cliff faces were not items crafted by some divine artisan but by natural forces, then appreciating them \*as if they were\* God-crafted artifacts, seems wrong-headed (cf. Carlson, 2000, Chapter 8). Similarly, if someone were to study a painting by Rembrandt, believing that it was in fact the product of natural forces slopping paint together, they would be seen as appreciating the object in question in a sub-optimal way (cf. Danto 1974, p. 140). In both cases, appreciation is undermined by a failure to recognise what the object in question really is. Different sorts of thing, Carlson says, require different modes of appreciation. Things like artworks and non-art artifacts, (e.g. laptops, hammers, washing machines), merit what he calls \*design\* \*appreciation\*. Things which are not designed, primarily for Carlson, the natural environment, warrant what he calls \*order appreciation\*. For both works of art and everyday objects, Carlson talks in terms of \*design appreciation\*. With paradigmatic artworks,\[^1\] we recognise them as creations of designers – objects where "every one of their features is the result of a decision by the artist" (Carlson, 2000, p. 109). Our appreciation centres on the relationship between the initial design and its embodiment: we consider whether the artist succeeded in their undertaking, how they worked with their materials, what constraints they faced, and whether the outcome realises their vision. This same approach extends to designed artifacts more generally. Carlson is explicit that functional objects are properly appreciated by seeing how their forms answer to what they are for: This is in part the point of the much-repeated phrase ‘form follows function.’ The forms of all functional objects – buildings, airplanes, and appliances as well as landscapes – must be aesthetically appreciated in terms of how and how well such forms fit their functions. However, the cliché is frequently interpreted too narrowly. With anything functionally designed, not only its form, but much of its aesthetic interest and merit, ‘follows function’. (Carlson, 2000, ch. 12, p.188). So a chair, a kettle, or a bridge invite the same style of attentive appraisal as a painting – guided by knowledge of ends, materials, constraints, and the fit between purpose and realisation. In \*order appreciation\*, we face objects that show order but have no designer behind them. Natural environments are the main case. Here there are no intentions to recover or evaluate. Instead, we find patterns and structures created by forces – geological, biological, meteorological – operating without purpose. Our task shifts from evaluating success against intention to understanding how these forces have shaped what we observe. Carlson describes its general form: On the assumption that order appreciation provides the correct model for the appreciation of nature, such appreciation has the following general form: An individual qua appreciator selects objects of appreciation from the things around him or her and focuses on the order imposed on these objects by the various forces, random and otherwise, that produce them. Moreover, the objects are selected in part by reference to a general nonaesthetic and nonartistic story that helps make them appreciable by making this order visible and intelligible. Awareness and understanding of the key entities – the order, the forces that produce it, and the account that illuminates it – and of the interplay among them dictate relevant acts of aspection and guide the appreciative response. (Carlson, 2000, p. 119) In design appreciation there is a split between the planner and the product: intentions, plans, and constraints precede and shape the artifact. In order appreciation there is no such split. In design, form precedes matter and is imposed upon it; in nature, order is immanent in the matter itself. In both modes, however, appropriate knowledge guides acts of aspection – what to look for, which dependencies matter, where to set boundaries, and how to draw contrasts (Carlson, 2000, p. 50). But the character of this knowledge differs. In designed cases, we need functional and technical understanding: what the designer intended and what constraints they faced. This knowledge shows us how ends and means relate. In natural cases, we need the appropriate scientific account – geomorphology, for instance, reveals how landforms develop over millennia. Any number of natural sciences might serve this role, and they are not mutually exclusive: the same landscape might be illuminated by geology, botany, and ecology together. Without such knowledge, natural structures might look accidental or chaotic; with it, we see them as effects of identifiable processes (Carlson, 2000, pp. 50, 60–61). Selecting a particular viewpoint or timeframe serves only to reveal the order more clearly, not to impose our own design. Once a specific scientific account is in play, some cases will show the relevant order better than others, preventing the worry that everything becomes equally appreciable (Carlson, 2000, pp. 118–119). The fundamental rule remains: do not project a planner where there is none; where something is made to a plan, judge it as such. It could be argued, however, that Carlson’s approach to aesthetics overlooks another important category of object of appreciation: people. In ordinary life we not only admire landscapes and artifacts; we admire people too – their wit, their manner, their steadiness. Some philosophers have taken this practice seriously, investigating the aesthetic appreciation of personality – sometimes termed “beauty of character” – and asking whether traits such as kindness, wit, or courage can be aesthetically as well as morally valuable (Gaut 2007; Paris 2018). Carlson's second recommendation seems naturally extendable here: appropriate aesthetic appreciation of persons will depend on the right kind of person-directed knowledge – familiarity with a life (real or fictional) and a sense of the values and dispositions that organise it. We do not admire kindness in the abstract, but this person's pattern of generous responses given who they are and what they have faced. As Parsons stresses, such knowledge is typically built up through direct interaction, careful biography, or the more precarious route of gossip (Parsons 2023, 297–299).\[^2\], \[^3\] Although Carlson does not consider person appreciation, it is not difficult to imagine ways in which his framework might be extended or modified to accommodate it. One option is to treat “persons” as a third category alongside natural items and artifacts, with their own distinctive mode of appreciation anchored in their status as subjects rather than as environments or tools. A second option is to treat the appreciation of character as a special case of order appreciation: we focus on the psychological, social, and biographical forces that shape a life, much as we attend to geological and ecological forces in a landscape. A third option would be to emphasise the ways in which personalities are, at least in part, self-shaped, and to appreciate them as self-designing projects – a thought that has obvious attractions for existentialist traditions. On all of these views, however, Carlson’s second recommendation still applies: aesthetic appreciation is guided by substantive background understanding of what persons are like and how their traits hang together over time. For present purposes, we need not decide which of these options is correct. It will be enough to note that person-based aesthetics, where it exists, presupposes a rich conception of the subject as a temporally extended agent with relatively stable dispositions, projects, and evaluative commitments, grasped under a suitable body of knowledge. In the rest of the paper, when we consider whether we can aesthetically appreciate LLMs “like people”, it is this sort of person-directed appreciation – and this Carlsonian constraint – that will be in the background. ## 2\\. What LLMs Are Carlson recommends we appreciate things for what they are. So what are LLMs? In this section we set the ground for appreciation by explaining the technical reality of these systems: how they process text as numerical tokens, calculate probabilities through learned parameters, and generate responses through iterative sampling. In later sections (§3 and §4) we use this reality – which differs in certain respects from that of traditional designed artifacts – to assess whether LLMs can be aesthetically appreciated as persons or as designed artifacts. Consider what happens when an LLM encounters the text "The cat sat on the". The system first breaks this into tokens-discrete units like words or word-parts. Importantly, each token is assigned a numerical ID (in this example, the ID numbers we use are just placeholders): ‘The' might become 464, 'cat' becomes 3857, 'sat' becomes 4521, and so on. The model works entirely with these numbers, not with words or meanings. It then assigns probabilities to possible continuations: token 5687 (which represents "mat") might have a 38% chance of appearing next, token 2931 ('floor') 22%, token 8104 ('chair') 15%, token 9823 ('roof') 8%, with thousands of other possibilities each assigned their own probability. The system does not simply pick the highest-probability token. Instead, it randomly samples from these probabilities. A parameter called \*temperature\* which can be set by the user controls how much randomness is involved. When temperature is set to zero, the model always picks the most probable token. This produces text that quickly becomes repetitive – the same phrases appearing again and again. When temperature is higher, around 0.8, the model sometimes picks less probable tokens. This leads to variation that looks creative. But it is randomness, not creativity. The model is rolling weighted dice, not making choices. The system selects one token – say "mat" – and appends this new token to create a longer sequence "The cat sat on the mat". It then calculates entirely new probabilities for what token should follow the extended sequence. Token by token, the system builds what appears to be coherent text through repeated numerical operations. No feasible amount of text could cover all the sequences the model might encounter, and storing all these combinations would be impossible anyway. Instead, the model learns general patterns during an initial pre-training phase: exposure to vast quantities of text – billions of pages from books, websites, and other sources – while learning to predict the next token in each sequence. The model begins with millions of numerical parameters (called 'weights') set to random values. Through repeated exposure, the system learns statistical regularities: which tokens tend to follow other tokens, which token sequences co-occur, how sequences typically unfold. The model stores these patterns as adjustments to its numerical parameters – decimal numbers that shape how strongly different tokens associate with each other. After seeing 'doctor' followed by 'patient' thousands of times, parameters adjust so that token 1245 ('doctor') increases the probability of token 7823 ('patient') appearing nearby. When the model wrongly predicts one token but the actual next token was another, the parameters shift slightly to make the correct token more likely in similar future contexts. After billions of such adjustments during pre-training, the model approximates the statistical patterns of human language. It does not learn that doctors treat patients or that cats are animals; it learns that, in the training distribution, certain number sequences follow others with certain frequencies. No programmer writes rules about grammar or meaning. The patterns emerge from exposure to text. The result is what is called a base model: a large, general-purpose text continuation engine. A key feature of this continuation engine is the \*embedding.\* In addition to its numerical ID, each token is represented as a vector – a list of numbers – that positions it in a high-dimensional mathematical space. Tokens that appear in similar contexts end up near each other in this space. 'Cat' sits near 'dog' because both appear after 'the', both can be followed by 'sleeps', both fit in phrases like 'fed my \\\_'. The model learns these positions through pre-training, not from programmed definitions. This is how meaning emerges in the model: not from understanding concepts but from tracking which words appear in similar contexts. The transformer architecture adds a mechanism called attention. This allows the model to connect related words even when they are far apart in a sentence. For instance, in 'The cat that chased the mouse sat on the mat', the model needs to know that 'sat' refers back to 'cat', not to 'mouse'. Through training, different attention heads can specialise in tracking different kinds of relationships. They do so by building specific representations not only for single words but also for sequences of words. Some attention heads track which pronouns refer to which nouns, others connect verbs to their subjects across long sentences. No one programmes these specific functions. They emerge because tracking these relationships helps minimise prediction error. In use, the model generates text through \*autoregressive decoding\*: each newly generated token gets added to the context, creating a new, longer sequence for which the model must calculate fresh probabilities. Given an input like 'What is the capital of France?', the model computes probabilities, selects token 464 ('The'), appends it to create 'What is the capital of France? The', recalculates probabilities for this new sequence, selects token 2341 ('capital'), and continues this mechanical process – 'The', 'capital', 'of', 'France', 'is', 'Paris' – until reaching a stopping point. Each step is purely computational: multiply numbers, add numbers, select token, repeat. The pre-training we have described so far teaches the model statistical patterns of language and yields a base LLM. In practice, most chat-oriented systems undergo a further \*post-training\* phase. After pre-training, the base model is fine-tuned on examples of instructions and responses, and then adjusted by RLHF, a process in which human raters evaluate the model's responses – rating them for helpfulness, accuracy, appropriate tone. The model then adjusts its parameters to produce more responses like those rated highly and fewer like those rated poorly. Post-training shapes the model's conversational norms: when to express uncertainty ('I'm not sure, but...'), when to decline requests ('I cannot help with...'), how to structure explanations ('Let me break this down...'). RLHF makes responses more consistent, more helpful, and more aligned with human expectations. But it operates through the same fundamental mechanism – adjusting numerical parameters to match patterns in the training data. The model learns which response patterns get high ratings, not why those patterns are appropriate or what social purposes they serve. The result is a \*chat-optimised\* model: the same predictive core, now biased towards a certain family of outputs that look like the moves of a cooperative assistant. When this chat-optimised model is embedded in a product – given a system prompt, safety filters, a memory policy, and a user interface – it becomes the chatbot that users encounter. What users describe as a model’s “personality” or “vibe” is a stable pattern in its responses under this post-training and product regime, not a separate mechanism or inner subject added on top of the predictive core. At several points in the preceding description, we saw that specific features of how LLMs behave are not programmed but emerge from training. Designers specify the architecture and training objectives, but the organisation of the trained system – the structures that underlie its behaviour – emerges from the training process. With a chair or a bridge, as we noted in Section 1, form precedes matter and is imposed upon it. With an LLM, designers create the conditions under which organisation will emerge, but they do not impose that organisation directly. Chris Olah, a co-founder of Anthropic, captures this vividly: one useful way to think about neural networks is that we don't program them... we don't make them... we kind of grow them... we have these neural network architectures that we design and we have these loss objectives that we create. And the neural network architecture, it's kind of like a scaffold that the circuits grow on... we create the scaffold that it grows on and we create the light that it grows towards. But the thing that we actually create, it's this almost biological entity or organism that we're studying. (Olah 2024) Carlson's recommendation to appreciate things for what they in fact are might seem to warn against taking such a comparison seriously – LLMs are not biological organisms, and their 'growth' is a computational process of parameter adjustment, not biological development. But the comparison is apt: the organisation of a trained neural network is not specified by its designers but emerges from a process they set in motion. This distinguishes LLMs from traditional designed artifacts, and, as we shall see, it has consequences for what kind of appreciation is appropriate. --- ## 3\\. Appreciating LLMs as Persons We sometimes appreciate persons aesthetically, responding to traits such as warmth, wit, or steadiness as ‘beautiful’ or ‘ugly’ features of character. Our appreciation of others goes beyond their physical appearance. You might admire or enjoy your friend's warmth or eccentricity, or a stand-up comic's quick wit, or a celebrity's self-deprecating demeanour; you might even appreciate the personalities of fictional characters: Gatsby's enigmatic, dream-chasing idealism; Ron Swanson's libertarian gruffness. It is therefore tempting to think that our appreciation of LLMs might be modelled on our appreciation of people. Many users already talk this way, describing their favourite models in terms of ‘personality’ or ‘vibe’. However, the technical description in the previous section presents LLMs as systems that tokenise text, manipulate numerical vectors, and generate continuations by sampling from learnt probability distributions, with a further post-training phase that biases them towards a certain assistant-like pattern of response. Nothing in that description straightforwardly resembles a subject with beliefs, intentions, or a life-history; there is no obvious place for character traits, projects, or personal development. Given Carlson’s recommendation that we should appreciate things as what they in fact are, and in the light of the right kind of knowledge, it is not yet clear that person-based aesthetic predicates are being applied to the right kind of object. In this section we ask whether, under that recommendation, there is any appropriate person-based aesthetic stance towards LLMs. We consider, in turn, make-believe approaches, concessive mindedness approaches, and a line of thought based on post-training and chat personae, and argue that none yields a satisfactory model of person-based aesthetic appreciation of LLMs themselves. ## 3.1 Make-believe approaches Start with the make-believe route. If we ask ordinary users whether they literally believe that a chatbot is a person, many will concede that they do not. They may talk to a model as if it were a friend or a colleague, and they may feel heard, reassured, or amused, but when pressed they acknowledge that they are interacting with a computational system rather than a human being. Their stance is, in this sense, already a kind of as-if posture. Mallory offers a way of theorising this posture through what he calls chatbot fictionalism (2023). On his view, we engage with chatbots by entering a game of make-believe in which the exchange is treated as if it were a conversation with an agent. Within the fiction, the chatbot 'says' things and 'means' things; outside the fiction, we know that no such speaker is present. At the metasemantic level, Mallory claims, the outputs lack literal semantic content – they are 'literally meaningless but fictionally meaningful' (Mallory, 2023, p. 1082). This fits the everyday thought that we can take a chatbot seriously in the moment without actually believing that it has a mind. Just as a child treats a banana as a sword in a game, we treat chatbot outputs as utterances within a kind of imaginative practice. This is not delusion but a deliberate, bounded pretence that allows us to coordinate with the system and even gain knowledge from it, much as we might learn geography from a map by imagining countries as two-dimensional shapes. Mallory’s account is not itself an aesthetics of LLMs; it is primarily a semantic and epistemic proposal about how we can use them and learn from them. But it highlights one obvious way a person-based aesthetic stance might be defended: one might suggest that we should aesthetically appreciate LLMs as if they were persons or characters, in the same sense in which we respond aesthetically to fictional protagonists whose existence we do not literally believe in. We respond to Gatsby’s enigmatic, dream-chasing idealism or Ron Swanson’s libertarian gruffness using much the same vocabulary as we use for real people, and we often talk quite straightforwardly about their ‘character’ or ‘personality’. In the fictional case, however, the protagonists are artifacts that have the function of eliciting imaginings of fictional persons within a story-world (cf. John 2021), so treating them as if they were persons does not misclassify their kind. Their role within the work is precisely to function as person-like figures in a narrative. By contrast, treating the LLM itself as a person would, given the architectural story in §2, amount to appreciating an artifact whose nature, as §2 stressed, is that of a large-scale text-prediction mechanism with post-training biases as if it were a subject with a life and character. LLMs do not call on us to imagine a fictional world inhabited by fictional characters but rather to consider the texts they produce as contributions to our inquiries. The function of mandating imaginings is not constitutive of generative AI in the way it is of fiction. Casting LLMs as fictional characters, in this sense, is a familiar kind of misclassification in Carlson’s terms. That is much closer to appreciating a mountain as if it were a divine sculpture despite knowing the geological story, and so sits badly with Carlson’s demand that appropriate appreciation respond to things as what they in fact are. Cross’s discussion of AI art systems develops something like this idea in the artistic context. He proposes what he calls the \*exploration paradigm\*, in which artists relate to AI systems as participants in a structured interaction: By adjusting inputs, iterating, and sampling, an AI artist is engaged in a process of mapping – and perhaps interrogating – the way that the algorithm sees and understands (Cross, 2025, pp. 7–8). Cross draws an analogy with performance art, where artists create spaces for audience participation. The AI artist's prompts structure a kind of 'participation' by the algorithm, and the resulting images serve as documentation of this exploration. But as Cross himself acknowledges, "the analogy... with performance art isn't a perfect one" (Cross, 2025, p. 9): AI cannot genuinely 'participate' since it lacks conscious choice or experience. What seems like participation is still statistical pattern-matching. While Cross's exploration paradigm offers a richer description of certain AI art practices than simple tool-use, it does not support person-appreciation for AI systems. The artist explores the algorithm's patterns, but the algorithm is not a participant in any literal or psychological sense. When Cross’s view is read as a model for our relation to the AI system itself, it looks like a kind of aestheticised make-believe. The artist is invited to treat the system as if it were a participant with a distinctive way of “seeing” or “understanding”, and the viewer is invited to regard the resulting interaction as a sort of joint performance. Mallory and Cross thus converge on a shared picture: in practice we often stand in relation to LLMs as if they were persons, and some of our aesthetic language is shaped by this as-if stance. Carlson’s recommendation now gives us a clear verdict on this first route. The as-if stance may be useful for interaction and may frame certain artistic practices, but an account of \*appropriate\* aesthetic appreciation of LLMs themselves cannot, on his view, rest on a stance that depends on systematically treating the object as something it is not. Once we have in view the technical reality described in §2, appreciating an LLM as if it were a person is analogous to appreciating a mountain as if it were a divine sculpture: it is to misapply person-based predicates to a case where the right background knowledge tells us that we are dealing with a different kind of thing. Make-believe personification may be harmless in some contexts, but under Carlson it cannot supply the correct mode of aesthetic appreciation for LLMs. ## 3.2 Concessive mindedness approaches If the make-believe route fails under Carlson’s recommendation, one might try a different strategy: instead of pretending that LLMs are persons, argue that they really are agents of a thin and unfamiliar kind. On a suitably liberal conception of mind, perhaps they qualify as intentional systems and that is enough to license some person-based aesthetics. Frankish (2024) offers a sophisticated version of this idea. Drawing on Dennett's intentional stance, he suggests that LLMs can be treated as genuine, if unusual, intentional systems. On this view, we are licensed to ascribe beliefs and desires to an LLM when doing so yields a simple and fruitful account of its behaviour, even if the underlying implementation is purely mechanical. In the case of contemporary chatbots, Frankish proposes that we can ascribe to them a large set of thin 'beliefs' – roughly, informational states distilled from their training – and one thin 'desire': to play what he calls the chat game. A system is playing the chat game when it generates text that looks like a cooperative move in an ongoing conversation, respecting local coherence, relevance to the prompt, and broadly human conversational norms. An LLM, on this picture, is a system whose behaviour can be summarised by saying that it believes many simple things and wants to make an appropriate next move in the chat. Crucially, this is not a make-believe view. Frankish is not inviting us to pretend that LLMs have beliefs and desires; he is claiming that, at the right level of abstraction, it is literally true that they do, in much the same sense in which a thermostat can literally be said to “want” the room to be at a certain temperature when adopting the intentional stance helps us describe its behaviour. The agent-talk is meant to latch onto real, pattern-like features of the system’s organisation. Suppose we grant all of this. Does it give us what we need for aesthetic appreciation of LLMs as persons? Here the benchmark sketched in §1.2 for person-aesthetics becomes relevant. A subject of beauty of character is not just any intentional system. It is, minimally, a being with a temporally extended life, with relatively stable value-laden dispositions, with projects and commitments that can succeed or fail, and with a capacity for speech and action to express and reshape its character over time. When we set the chat-game agent against this benchmark – the conception of persons implicit in beauty-of-character talk sketched in §1.2 – it looks thin. The ‘beliefs’ are shallow, in the sense that they are confined to what is encoded in the model's parameters and surfaced in the current context, without memory or development across conversations. The "desire" is singular and thin: make an appropriate move now in this exchange. There are no independent projects pursued across episodes, no webs of concern or attachment, no history in which earlier experiences inform later choices. What structure there is, is entirely local to the present stretch of text. The predicates characteristic of person-aesthetics—'beautiful soul,' 'admirable steadiness,' 'ugly character'—presuppose something that can be tested, developed, or refined over time; a thin chat-game agent has no such temporal depth. From this perspective, LLMs may be agents in Frankish’s concessive sense, but they are not the sort of agents whose lives and characters can be the object of the aesthetic responses associated with persons. There is nothing like a “beautiful soul” or an “ugly character” here in the relevant sense; there is no enduring set of values and dispositions that could be manifest, challenged, or transformed over time. Given Carlson’s recommendation, the right kind of person-directed knowledge for beauty-of-character appreciation is knowledge of a life and its values. The technical and training facts about LLMs do not supply that kind of object. Once again, Carlson’s recommendation sharpens the point. If we accept the technical story about the real nature of LLMs (see §2) and, even on a concessive mindedness view, we see that LLMs lack the life-structure required for person-aesthetics, then to insist on aesthetically appreciating them as persons would be to ignore what they in fact are. It would be to treat the thin chat-game profile as if it were enough to underwrite the rich person categories we apply to human agents, and to let those categories govern appreciation despite knowing that the underlying kind is different. The concessive strategy therefore does not secure an appropriate person-based aesthetics of LLMs. Taken together, then, the make-believe and concessive-minded strategies cover the most natural ways of defending a person-based aesthetics of LLMs. The first tells us to appreciate them as if they were persons, despite knowing that they are not; the second tells us that they really are agents of a thin sort but does not supply the temporal and evaluative structure that person-aesthetic predicates require. Under Carlson’s framework, neither route yields a correct model of how LLMs should be aesthetically appreciated. ## 3.3 Post-training, chat personae, and thin agency A natural objection at this point is that these arguments underplay the role of post-training and the chat interface. Section 2 noted that base models are further fine-tuned on instructions and shaped by RLHF, and that the resulting chat-optimised systems exhibit stable patterns of hedging, refusal, politeness, and explanatory structure. One might suggest that this post-training regime turns bare LLMs into conversational agents and that, under Carlson’s recommendation, we should take those chat assistants as the “things as they in fact are” and allow some form of person-based aesthetic stance. The technical story in §2 suggests a more layered picture. On the one hand there is the underlying generative system: the predictive core that, after pre-training, approximates the statistical structure of its training corpus and that, after post-training, remains a text continuation engine with a modified probability landscape. On the other hand, there are patterns in its outputs that, under chat-style prompting and within a product wrapper, look like the moves of a cooperative assistant persona. The assistant is not a new mechanism added on top of the model, but a recurrent pattern in how the model tends to respond when prompted and constrained in certain ways.\[^4\] Seen in this light, post-training does not install a new “assistant mind” with its own independent goals and projects. It biases the predictive core so that prompts issued through the chat interface are much more likely to elicit assistant-like responses – helpful, safe, polite, and structured – and much less likely to elicit, for example, unfiltered reproductions of online arguments or free association. The underlying operation remains next-token prediction; what changes is which regions of its behavioural space are easy to reach in ordinary use. The chat product – with its system prompt, safety filters, and interface – further shapes the environment so that certain person-like patterns are the default. This helps explain why users talk about models having different ‘vibes’. If users say that Claude Opus 4.5 feels friendlier than GPT-5.2, they are picking up on a stable pattern in how the chat-optimised systems tend to respond across many prompts and episodes. They track which assistant personae tend to appear and how those personae typically behave – not a unified character with a life and projects. Different base models, post-training regimes, and product designs favour different families of assistant-style responses. It is therefore not surprising that they invite person-like language, but the targets of that language are episodes and recurring response profiles, not underlying subjects. One might find a particular model’s refusals laboured or concise, its hedging overdone or judicious, its tone soothing or dry. In that sense, we can aesthetically respond to assistant personae in a way that resembles our responses to real people. What matters for present purposes is that such reactions target patterns in outputs and interactional style, not a subject with a life (in the sense sketched in §2). They concern how a product behaves under certain constraints, not the beauty or ugliness of a character in the person-aesthetic sense. If we ask instead about the generative system itself – the predictive core tuned by post-training and embedded in a chat product – the earlier verdict remains. Even taking post-training and chat personae fully into account, we do not find a temporally extended life, a network of projects and commitments, or a stable evaluative outlook that could ground beauty-of-character predicates. What we find is a complex artifact designed and trained to produce certain patterns of text in response to prompts, together with an engineered tendency to exhibit assistant-like behaviour in a controlled range of contexts. Under Carlson’s recommendation to appreciate things as what they are, and in the light of the right kind of knowledge, we should therefore resist person-based aesthetics for LLMs even once we take post-training and chat personae into consideration. As Farrell, Gopnik, Shalizi, and Evans (2025) put it, “Large models should not be viewed primarily as intelligent agents but as a new kind of cultural and social technology, allowing humans to take advantage of information other humans have accumulated”. Thus, in the rest of the paper we set aside person-based aesthetics and turn to these alternatives: first, treating LLMs as designed artifacts and considering the limits of simple “form follows function” stories (§4); then, developing an order-based mode of appreciation that focuses on the text mechanics that characterises chat instances (§5–§6). --- ## 4\\. Appreciating LLMs as Artifacts Having set aside the person-based options in Section 3, we turn to design appreciation. Contemporary LLMs are artifacts: they are built and deployed by corporations and research groups, engineered to satisfy aims such as helpfulness and safety, and revised in light of user feedback and product strategy. Given Carlson’s emphasis on artifacts and design appreciation, it is natural to ask whether we should aesthetically appreciate LLMs as designed tools, asking how well their forms serve their functions. On this view, models such as GPT-5.2, Claude 4.5 Opus, and Gemini 3 Pro look like canonical objects for design aesthetics: complex, purpose-built systems whose architecture, training recipe, and user interface might be admired for elegance, efficiency, or ingenuity. Existing work on the aesthetics of design develops this general thought. Carlson notes that, for objects that are designed to perform some task, their forms “must be aesthetically appreciated in terms of how and how well such forms fit their functions”, and he glosses the familiar slogan “form follows function” by adding that, with anything functionally designed, “not only its form, but much of its aesthetic interest and merit, ‘follows function’” (Carlson 2000, chapter 12). Forsey’s Kant-inspired account of design as a case of dependent beauty and Parsons and Carlson’s later theory of functional beauty can both be read as ways of spelling out this claim. Forsey argues that judgements of design beauty presuppose a concept of what the object is meant to be and do, and that our grasp of its success in fulfilling that role informs the aesthetic verdict itself rather than merely accompanying a “pure look” at its lines (Forsey 2013). Parsons and Carlson explain how knowledge of function can structure experience so that an artifact’s form can be experienced as fit, streamlined, overbuilt, and so on, yielding functional beauty when the form presents itself as well suited to what the thing is for (Parsons and Carlson 2008, chapter 4). Taken together, this cluster of views treats appropriate design appreciation as a matter of aesthetically responding to how a functional artifact is put together to do what it does. From this standpoint, it is natural to try to assimilate LLMs to the design template. In a given deployment, the artifact can be characterised by a relatively unified functional role – for example, that of a general-purpose conversational assistant embedded in other tools – and by a specific way of realising that role through architecture, training, and alignment. Under that description, much of what seems aesthetically salient about a deployed model concerns how its engineered “form” serves that function: whether its interaction profile is cluttered or economical, whether it sustains a clear argumentative line or habitually wanders, whether refusals and clarifications are integrated smoothly into the exchange or arrive as abrupt blocks, whether long-context processing and tool-calls are handled in a way that keeps the conversation legible. Forsey’s dependent-beauty framework and Parsons and Carlson’s functional-beauty account can be used to gloss such assessments: they remind us that any appraisal of design beauty here presupposes a concept of the assistant’s role and some understanding of how that role is realised in the artifact’s structure and behaviour. At the same time, both accounts were developed for cases in which the relevant form is a stable, visible configuration – buildings, bridges, bicycles – where function can literally show up in perceptual appearance. In the LLM case, by contrast, the structures that realise the assistant role are not perceptually available in this way, and the aspects that prove most revealing are not static shapes but patterns in generated text over time. This already limits the reach of straightforward “form follows function” stories for LLMs and points towards a more order-centred mode of appreciation. With traditional designed artifacts, design-knowledge illuminates structure because designers specified it. Knowing what the designer intended and what constraints they faced helps us understand why the artifact has its form – even for structural features that are not directly visible, such as a bridge's internal stress distribution. With an LLM, the situation is different in kind. The organisation of the trained system – as Section 2 established – emerges from training rather than being specified in advance. Design-knowledge therefore does not illuminate this emergent organisation: there was no designer's specification that laid it out. To understand it, one must attend to the training process that produced it. LLMs are artifacts, so design-knowledge is not wholly without application. We can appreciate how the architecture is suited to the function, how post-training shapes conversational behaviour, how the interface presents the system to users. But if the aesthetically revealing features arise from emergent organisation rather than from the designed scaffold, design-knowledge alone will not suffice. We also need knowledge of the processes that produce the emergent order. The thought that appreciating certain artifacts requires knowledge beyond design-knowledge is not unique to AI. Ceramic traditions such as raku and wood-fired pottery make this explicit. The potter shapes the vessel and chooses the glaze, then yields to kiln, flame, and ash; the maker harnesses but does not micromanage these forces, and they finish the surface in ways no blueprint prescribes. Appreciating such a bowl requires knowledge of both the potter's choices and the kiln processes. Pollock’s action paintings occupy a similar hybrid space within the art domain. Carlson uses them to illustrate how order appreciation can depend on knowledge of the forces at work: “awareness and understanding of \\\[natural\\\] forces is vital in nature appreciation, as is knowledge of, for example, Pollock’s role in appreciating his action painting or the role of chance in appreciating a Dada experiment.” Pollock chooses canvases, pigments, and tools, and choreographs his movements over the surface; yet gravity, viscosity, surface tension, and drying behaviour make a substantial contribution to the patterns that settle. To appreciate a Pollock appropriately, on Carlson’s view, is not just to admire his intentions; it is to attend to the order produced by the interplay of deliberate gesture and physical process, informed by an understanding of the role of chance and material behaviour. In all three cases, namely Raku ceramic, Pollock’s action painting, and LLMs, the maker creates conditions and then yields to kiln-fire, gravity, and training dynamics respectively. Forces beyond specification complete the work. What matters aesthetically is the emergent completion, not only the scaffold that enabled it. Section 2 emphasised that during training the model places tokens in a high-dimensional space on the basis of contextual co-occurrence, that attention mechanisms self-organise to track different sorts of dependency across context, that different layers specialise in local or global patterns, and that RLHF shapes an interactional style by rewarding some forms of response and penalising others. None of these details are written into the code as explicit rules about how to, say, handle metaphors, or politely decline illicit requests. They are emergent regularities in a trained network that has been pushed, by the neural network and its training data, to reduce prediction error. What grows on Olah’s “scaffold” is, in practice, a system of statistical associations and processing circuits whose internal organisation even designers often understand only partially. From a design-aesthetic perspective, this matters. Much of what users find \*aesthetically\* important in LLM behaviour – the way a model sustains a metaphor or abruptly drops it, the pattern of hedging and self-correction, the texture of its reasoning, the sorts of digression it tends to indulge, the characteristic “feel” of its refusals – is grounded in features (concerning embeddings, attention patterns, layer dynamics, and RLHF) that have not been micro-designed but have emerged from optimisation under constraints. Parsons and Carlson note that, even for simpler artifacts, knowledge of function must include knowledge of how that function is realised if it is to structure perception appropriately. In the LLM case, knowing that “this is a general-purpose assistant” is not enough to make sense of its aesthetic profile; what does the work is knowledge of the way training and alignment have \*grown\* a particular style of continuation on top of the architecture described in Section 2. In this sense, knowledge of how function is realised concerns \*growth\* rather than \*design\*. This hybrid status complicates simple appeals to “form follows function”. On the one hand, some design-appreciative predicates apply straightforwardly. It makes sense to say that a model whose interface is cluttered or opaque is, as a product, less well-designed than a lean one; it makes sense to prefer an alignment regime that avoids gratuitous scolding or needless refusals; it makes sense to admire a training setup that achieves a good balance between fluency and factual reliability. Forsey’s notion of teleological style can be extended here: different labs realise the shared function ‘LLM assistant’ in recognisably different ways, and those ways can be compared and assessed. Parsons and Carlson’s notion of functional beauty also has a foothold: understanding how an LLM’s architecture supports its function can inform our appreciation of the system’s efficiency, robustness, or clarity as an artifact. On the other hand, if we try to make design appreciation do all the work, we mislocate the primary source of what matters aesthetically. In the chair or bicycle case, the designer’s choices fix most of what matters aesthetically: small emergent contributions from wear, patina, or use sit on top of a tightly specified plan. In the LLM case, by contrast, the order that matters aesthetically is largely the order of a trained statistical system running under its own learned constraints. Designers specify objectives and scaffolds but the particular ways in which embeddings cluster meanings, attention heads track long-range connections, layers distribute processing, and RLHF imprints a “vibe” are not written down anywhere as a plan. These are closer, structurally, to the ash-produced flashes on a raku bowl or the tangled skeins of a Pollock surface than to the thickness of a table leg or the proportion of a doorway. The upshot is modest but important. LLMs are artifacts, and there is a place for design appreciation in their aesthetic appraisal: we can and should evaluate how well their forms answer to their engineered functions, as well as how this form can differ from model to model However, the most distinctive and revealing aesthetic phenomena arise not from the execution of a detailed design, but from the emergent linguistic order that these grown systems exhibit when they are run. To appreciate that order, we need knowledge not of what designers intended but of how training shapes text propagation—what we call \*semiotic physics\*.. --- ## 5\\. Semiotic Physics ## 5.1 Textual Regularities Section 2 described what LLMs are: token-based predictors trained on large text corpora and shaped by RLHF. This satisfies Carlson's first recommendation: appreciate things as what they are. The second recommendation requires the right kind of knowledge to guide aspection. For LLM outputs, what knowledge makes their patterns visible and intelligible? Several sub-disciplines of computer science might be candidates. One field that has emerged in connection with neural networks is mechanistic interpretability, which investigates the internal workings of these systems by identifying which circuits, attention heads, and internal representations handle different linguistic tasks (Olah et al. 2020; Elhage et al. 2021). This work yields knowledge of how LLMs operate. But mechanistic interpretability functions at a level that requires specialist tools to observe. Its objects of study – weight matrices, activation patterns, circuit-level features – are not available to readers encountering generated text. Consider the difference between chemical physics and geology when appreciating a cliff face. Chemical physics provides knowledge of molecular bonds within rock, but it operates at a scale invisible to the naked eye. Geology, by contrast, offers concepts – strata, faults, erosion channels – that connect to what can be seen. One can perceive strata without specialist equipment, and knowing how sedimentation works makes the visible layering intelligible. Mechanistic interpretability faces a parallel limitation: while it reveals internal mechanisms, its objects of study are hidden from the user reading generated text. For an aesthetics of LLM outputs that is accessible to ordinary users, we need a framework whose concepts describe perceivable features and render them intelligible as products of the system's learned regularities. The forces of semiotic physics are not alternative explanations to those of mechanistic interpretability but the same processes described at the level at which they produce perceivable linguistic order. Recent work on LLMs points towards such a framework. Janus (2022) proposes that GPT-style models are best understood not as agents or oracles but as simulators: systems that have learned to propagate text according to regularities induced from training data. The model learns what Janus calls 'the conditional structure' of its training distribution – patterns governing what tends to follow what under what conditions. The analogy to physics is explicit: just as physical laws describe regularities governing what happens under given conditions, the trained model embodies learned regularities governing how text continues from any starting point. A prompt specifies initial conditions; the model propagates text forward according to its learned regularities. Picca (2025) arrives at a convergent view from a semiotic perspective. LLMs are "semiotic machines" that "recombine, recontextualize, and circulate linguistic forms based on probabilistic associations" (Picca 2025, 1). The emphasis shifts from internal mental states to patterns of sign-transition. The term 'semiotic physics' emerges from subsequent discussion of Janus's work (Kirchner 2023; metasemi 2023), naming the study of how intelligible text arises from sub-semantic processes—the regularities governing text propagation in trained language models. Despite their different framings – Janus's simulator ontology and Picca's Peircean semiotics – these accounts share a core insight: we should attend to what regularities govern how text propagates through the system, not to whether LLMs think or intend. In a similar vein, Wolfram (2023) states that inside ChatGPT any piece of text is effectively represented by an array of numbers that we can think of as coordinates of a point in some kind of ‘linguistic feature space’. So when ChatGPT continues a piece of text this corresponds to tracing out a trajectory in linguistic feature space. But now we can ask what makes this trajectory correspond to text we consider meaningful. And might there perhaps be some kind of ‘semantic laws of motion’. From Wolfram’s perspective, semiotic physics thus would have three main objects to investigate: (i) the “linguistic feature space” in which words and other linguistic items have their place; (ii) the “trajectories” that can be traced out in this space to continue a piece of text; and (iii) the “semantic laws of motion” that determine such trajectories. We draw on this literature but develop it in a specific direction. Our aim is to show how semiotic physics can serve as the "right kind of knowledge" for aesthetic appreciation of LLMs in Carlson's sense: the knowledge that makes order visible and intelligible, and that guides acts of aspection. The connection to environmental aesthetics, and the claim that semiotic physics can play the role for LLMs that geology plays for landscapes, is our contribution. We also articulate the 'forces' of semiotic physics at the level of textual effects rather than at the level of mechanistic detail. The existing literature tends to discuss semiotic physics in terms of probability distributions, embedding spaces, and dynamical systems. These descriptions are accurate, but they do not directly connect to what readers can perceive in generated text. Our articulation of the forces operates at a level that does connect to perceivable features. What does semiotic physics track? The regularities it describes manifest as perceivable features of generated text. Consider vocabulary clustering: words do not appear independently but make other related words more probable, so that once a medical term appears, other medical terms become more likely to follow. Or consider coherence dynamics: the model threads material from earlier in an exchange through later responses, or fails to, and a reader can attend to how far this threading extends and where it breaks down. There is also what might be called register stability: once the model enters a mode – expository, creative, reasoning – it tends to remain there until something disturbs it. And there are the marks of post-training: hedging expressions, step-by-step organisation, preemptive qualifications, which are the shapes that reinforcement learning has made more probable. What matters for present purposes is the level of description: semiotic physics operates at a level that connects to perceivable features of language, features that competent readers can attend to without specialist tools but that become salient and intelligible when understood as products of a text-trained statistical system. According to Wolfram (2023), LLMs reveal that "human language (and the patterns of thinking behind it) are somehow simpler and more ‘law like’ in their structure than we thought. ChatGPT has implicitly discovered it. But we can potentially explicitly expose it". Semiotic physics pursues such an exposition by investigating the forces that govern the artificial production of texts. One might object that speaking of 'forces' in relation to LLMs is metaphorical in the same way that speaking of 'agents' or 'intentions' is metaphorical. If we have rejected agent-talk as projecting non-existent mental states onto a statistical system, why is force-talk any better? The answer turns on a distinction between metaphorical personification and literal causal abstraction. To speak of an LLM as an 'agent' is to attribute to it internal states – intentions, beliefs, a 'self' – that play no role in its functional operation. To speak of the forces of semiotic physics is to identify the factors that determine the selection of each token. These are not projected onto the system; they describe what the system does. Assuming Wolfram’s (2023) characterization of the continuation of a text by a LLM as “tracing out a trajectory in linguistic feature space”; the forces of semiotic physics are literally the causal factors that determine that trajectory, just as mechanical forces determine the trajectory of a material body in physical space. The template for this literalism is in Carlson's analysis of Jackson Pollock's action paintings. Carlson argues that we appreciate a Pollock not by looking for a designer's plan but by focusing on the order imposed by "the internal dynamics of his material": "the viscosity of the paint, the speed and direction of its impact, the interaction with other layers of pigment" (Janson, quoted in Carlson 2000, 111). For Carlson, these are not metaphors borrowed from a physics textbook; they are causal factors that produce the pattern on the canvas. In the semiotic environment of an LLM, semantic attraction and modal inertia play the role that viscosity and gravity play in Pollock: they are determinants of how text propagates; once entered in a given discursive mode, the model tends to stay in this mode. By identifying them as 'forces', we are describing the system as a productive mechanism in naturalistic terms. Knowing that a text is LLM-generated rather than human-written changes how we aspect it. This mirrors the shift that occurs when someone moves from believing that a cliff face was crafted by a divine artisan to understanding it as a natural formation. The visual field is the same, but aspection differs. When we believe in the divine artisan, we attend to the composition as evidence of design choices: the placement of features, the aesthetic arrangement. When we understand the geological story, different features become salient: strata as traces of sedimentation, erosion channels as marks of water flow, fault lines as evidence of tectonic forces. We stop attending to intentional composition and start attending to the marks of natural processes. For LLM text, the analogous shift is from reading as expression of an author to reading as product of semiotic forces. When we read a text as human-written, we attend to authorial intention (what is this person trying to communicate?), individual voice (what is distinctive about how this person writes?), and biographical traces (what does this reveal about the author?). When we read a text as LLM-generated, with knowledge of semiotic physics, different features become salient: vocabulary clustering as the mark of semantic attraction, coherence dynamics as the mark of contextual threading, and response structure as the mark of alignment pressure. The same words on the page; a different aspectual focus. The aspection guided by semiotic physics is, in a sense, aspection of language itself – of the textual order produced by semiotic forces. We are not attending to mechanical internals – activation patterns, attention weights, circuit-level features – since these require specialist tools and are not accessible to readers. We are attending to the textual manifestation of semiotic order: how vocabulary clusters, how coherence is maintained or lost across an exchange, how register persists or shifts, how post-training shapes response structure. These are features of the language itself, perceivable by competent readers. Competent readers already have tacit knowledge of how language works: syntactic, semantic, pragmatic, and discourse-level knowledge built up through immersion in spoken and written language. They perceive patterns in LLM outputs using this tacit knowledge. Semiotic physics adds explicit articulation of these patterns and a causal story about their source in training. The competent reader senses that different models have different 'vibes'; semiotic physics—knowledge of how meaning clusters, how register persists, how training shapes the texture of response—explains what produces those vibes and makes them available for sustained attention. ## 5.2 Practical Acquaintance Semiotic physics, articulated as an explicit theoretical account, is one way of holding the knowledge that guides appreciation. But Carlson notes that scientific knowledge and common, everyday knowledge of nature lie on a continuum rather than being different in kind. Both can guide appreciation of natural order. The farmer, the gardener, and the forester know the land through working it. Their knowledge is not typically framed in scientific vocabulary, but it is knowledge of natural order. The farmer knows the soil through planting, tending, and observing how different crops respond under different conditions. Through repeated intervention and observation, the farmer builds up knowledge of the regularities at work: drainage patterns, soil composition, seasonal cycles. This practical knowledge can guide aesthetic appreciation. The farmer may appreciate the order in a well-drained field, or the texture of properly cultivated soil, in ways unavailable to someone who merely gazes at the landscape. The knowledge is not scientific in the technical sense, but it connects to perceivable features and makes order visible and intelligible. The experienced user of an LLM develops analogous practical acquaintance. By prompting, experimenting, and observing how a system responds across many contexts, users build up knowledge of its characteristic order. They learn which semantic attractors the model falls into: which vocabulary clusters it tends towards given certain starting points. They learn how far contextual threading extends: at what point the model loses track of earlier material. They learn what triggers mode shifts: what kinds of prompts push the model from expository mode to creative mode, or from helpful mode to refusal. They learn the characteristic shapes that alignment pressure produces: the hedging rhythms, the step-by-step structures, the politeness markers. This is knowledge of semiotic physics held practically rather than theoretically. The experienced user cannot necessarily articulate the forces explicitly, but they have a feel for how the model behaves – expectations that are predictive (what kinds of outputs to expect) and aspectual (what to attend to, which features are salient, where to look for the model's characteristic order). Extended exchanges with an LLM are a natural site for this interactive mode of appreciation. Prompting is intervention; responses reveal regularities. Each turn creates conditions under which the system responds, and the responses reveal something about the model's semiotic physics. The back-and-forth of prompting is itself a mode of aspection. It selects what to attend to, organises appreciative attention over time, and tests and refines the user's developing sense of the model's characteristic order. Cross (2025) characterises certain AI art-making activities as an "exploration paradigm" in which the artist iteratively probes the model, adjusting prompts and sampling variations. Section 3 was critical of reading this as literal collaboration between artist and algorithmic "participant". From the present vantage, however, the practice can be reinterpreted. What the artist is doing, when things go well, is a form of interactive aspection: using structured engagement to reveal and respond to the model's characteristic order. The prompts and adjustments are not ways of coordinating with a co-creator; they are ways of making the system's semiotic regularities visible. The explicit theoretical account of semiotic physics and the practical acquaintance built through interaction are continuous. The farmer's knowledge of the land and the geologist's knowledge track the same forces – geological, hydrological, ecological – operating at the same scales. They differ in how the knowledge is held and articulated, not in what it is knowledge of. The experienced LLM user's practical sense of how a model behaves and the theorist's account of semiotic physics track the same regularities: semantic attraction, contextual threading, modal inertia, alignment pressure. Both routes converge on the same object: the model's characteristic semiotic order. Both guide the same kind of aspectual attention: attention to how semiotic forces have shaped the text. Whether held theoretically or acquired through practice, knowledge of semiotic physics makes the order in LLM outputs visible and intelligible and guides the acts of aspection appropriate to appreciating that order. ## 6\\. Levels of Appreciation LLMs can be appreciated at three levels: individual outputs, extended chats, and models themselves. Discussion of generative AI aesthetics has so far focused on outputs, such as images from Midjourney and texts from ChatGPT. But chats and models are also objects of appreciation, and the framework developed in Section 5 applies at each level. The relations among these levels can be clarified by analogy. An individual output is like an individual natural object, a tree, say: it is a sample of how semiotic forces have shaped a particular text under particular conditions. A chat is like an environment, a forest: semiotic forces shape the exchange over many turns, producing a configuration with its own coherence and dynamics. A model is like a natural system, the planet's biosphere, or the planet itself: it is the ground of order that manifests in outputs and chats, the system whose regularities produce those manifestations. Appreciation at each level calls for its own acts of aspection, though all are guided by knowledge of semiotic physics. ## 6.1 Appreciating Outputs A single output is one realisation of the model's semiotic physics under particular conditions. The prompt, the system configuration, and the preceding context specify initial conditions from which the model propagates text in line with its learned regularities. Different prompts activate different regularities; different contexts produce different trajectories. No single output exhausts the model's characteristic order. But each output shows how the forces operate in a specific case, and each can be appreciated as such. To show how semiotic physics guides aspection of outputs, we consider two cases that occupy different regions of a model's behavioural space. The first is the reasoning-style output familiar from everyday use: step-by-step structure, numbered stages, explicit hedging, restatement of the question, and a concluding summary. As human prose, such passages resemble competent but unremarkable textbook writing. They are useful for teaching and troubleshooting, but they do not obviously invite aesthetic attention. From the perspective of semiotic physics, however, the same outputs look different. The model has been trained on reasoning-related texts: worked proofs, textbook explanations, exam solutions, and online Q&A threads. It has learned that certain kinds of questions are typically followed by sequences with a characteristic structure. Post-training procedures, including instruction tuning and reinforcement learning that rewards explicit intermediate steps, further bias the model towards this pattern. Reasoning-style outputs are a stable attractor in the model's behavioural space: once entered, the model tends to stay in this mode, yielding modal inertia. The hedging, the step-by-step structure, and the summary are marks of alignment pressure: response shapes reinforced because they correlate with high human ratings. Given this, we attend differently. We attend to the characteristic rhythm of the reasoning mode: how steps are sized, how transitions are signalled, and whether the pacing is tight or padded. We attend to where alignment pressure shows: hedging patterns ('it seems', 'one might argue', 'I think'), politeness markers, and pre-emptive qualifications. We attend to how semantic attraction operates under tight constraints: vocabulary stays on topic, related terms cluster, and the model is pulled towards the semantic field established by the question. We also attend to whether the mode remains stable or shows signs of strain, and to whether the model sustains the reasoning register or begins to drift. What seemed merely useful becomes appreciable as a specimen of how semiotic forces produce reasoning-like text under tight constraints. The second case is different. The text discussed here was produced by a Claude-like model in a modified configuration with safety constraints relaxed. It begins with neologisms and proceeds in short blocks separated by headings in capitals. The vocabulary is dense with coinages, many of which recombine recognisable roots from entomology, anatomy, theology, and internet slang. The registers are mixed: fragments of cod-French, pseudo-scientific talk, mystical declarations, and obscene slang. Despite the surface disorder, a stable theme runs throughout: bees and honey, tongues and throats, sweetness, bodily contact. Under semiotic physics, this text shows the forces operating under loose constraints. Semantic attraction is at work: the bee and honey theme creates an attractor, and related vocabulary – tongues, throats, sweetness, pollen, flowers, stings – is pulled towards it. But unlike the reasoning case, the attraction spreads freely across registers rather than being channelled narrowly. The model has been trained on texts that invent words – experimental poetry, surrealism, internet wordplay – and it has learned patterns of neologism: how to recombine roots, suffixes, and sound-shapes. The neologisms follow learnable patterns of word-formation rather than being random noise. The register collision reflects training diversity: the model has absorbed texts in many registers (scientific, mystical, erotic, internet-surreal), and under loose constraints these do not get filtered to a single appropriate register. They collide and mix. Despite the apparent chaos, there is order: recurring rhetorical templates, alternation between narrative stretches and reflective sentences, and consistent sound-play in the neologisms. This order is the product of semiotic forces operating with fewer constraints than in the reasoning case. Attending to this text with knowledge of semiotic physics, we notice how semantic attraction shapes the vocabulary: the gravitational pull towards bee-related terms operates across registers. We also notice patterns in neologism (learnable word-formation rules that produce coinages with a family resemblance) and the rhythm of alternation between modes (narrative stretches, reflective sentences, and exclamatory outbursts). Finally, we notice internal consistency despite surface chaos. The text becomes appreciable as a specimen of semiotic forces operating in a different region of behavioural space from the reasoning output. Carlson (2000) notes that once a specific scientific account is in play, some natural formations show the relevant order better than others: not every cliff face is equally instructive about sedimentation, not every valley equally revealing of glacial dynamics. This prevents order appreciation from collapsing into the view that everything is equally appreciable; the guiding knowledge discriminates among cases. The same holds for semiotic physics. Standard reasoning-style outputs show semiotic order, but the order they show is shallow and familiar: alignment pressure is everywhere visible, the reasoning template is stock, and the semantic channelling narrow enough that the regularities are unsurprising. The bee text is a more interesting object of appreciation not because it is more orderly but because it reveals order where none was expected. What looks like chaos – neologistic excess, register collision, surface incoherence – turns out, under semiotic physics, to be structured by identifiable forces: semantic attraction spreading freely across registers rather than channelled narrowly, learnable word-formation patterns producing coinages with family resemblance, rhythmic alternation and internal consistency maintained beneath apparent disorder. The bee text also shows forces operating in regions of behavioural space that normal product configurations occlude. It is, in this sense, analogous to a geological formation that exposes strata usually buried – not more ordered than the surrounding terrain, but more \*revealing\* of the order that is everywhere present. The contrast between these two cases helps to locate what semiotic physics brings into view. Reasoning outputs show semiotic forces operating under tight constraints: a stable mode, narrow semantic channelling, and alignment pressure shaping response structure. The bee text shows semiotic forces operating under loose constraints: unstable modes mixing, semantic attraction spreading across registers, and training diversity showing through. Both are products of the same semiotic physics, but they occupy different regions of the model's space. Appreciating both requires the same kind of knowledge – knowledge of semiotic forces – but different acts of aspection. We scan the reasoning output for rhythm and regularity; we scrutinise the bee text for pattern within apparent chaos. ## 6.2 Appreciating Chats Carlson's environments are not collections of discrete objects but systems in which forces operate and interact over space and time. A forest is not just many trees; it is a space where ecological forces – competition for light, nutrient cycling, succession dynamics – play out, producing emergent order that no single organism embodies. The appreciator navigates this environment, and their path determines what order becomes visible. Chat instances stand to single outputs as environments stand to individual natural objects. A chat accumulates context that shapes how semiotic forces manifest: early vocabulary choices establish attractors that persist, early register-setting constrains later exchanges, and the exchange develops path-dependent structure that neither party fully controls. The user's prompts are not just elicitations but navigational interventions, steering the system through different regions of its behavioural space and making different orders visible. To appreciate a chat is to appreciate an emergent configuration produced by semiotic forces operating over the chat's temporal extension – not just a sequence of isolated responses. A single output is one trajectory from one set of initial conditions. An extended exchange lets regularities show up across turns. The model carries forward elements of earlier responses, picks up threads, sometimes drops them, and shifts register in response to user prompts. Chats manifest features that a single output does not. First, coherence maintenance, that is, how the model sustains or loses threads across turns, how far back its effective 'memory' extends, and where coherence begins to fray. Second, context accumulation, which determines how earlier material shapes later responses, and how terms or framings established early persist or fade. Third, register dynamics, which concerns how the model responds to shifts in user tone, topic, or style, and whether it matches the user's register or maintains its own. Finally, mode stability over time, to wit, whether the model stays in a mode or drifts, what triggers transitions, and how gracefully it handles them. These are manifestations of semiotic forces operating over longer timescales than a single output can reveal. A chat instance is like a particular forest: the forces of semiotic physics have produced a specific configuration. Different prompting strategies, different topics, and different user styles produce different configurations. But the same underlying forces are at work. Appreciating a chat means attending to how the forces have shaped this particular extended exchange: how contextual threading has produced coherence or incoherence, how modal inertia has maintained or failed to maintain a register, and how alignment pressure has shaped the arc of the exchange. In a chat, prompting is intervention. Each turn is a probe that reveals something about the model's regularities. The experienced user's expectations are tested and refined across many turns. Interaction is itself a mode of aspection: it selects what to attend to, organises appreciative attention over time, and deepens practical acquaintance with the model's semiotic physics. The farmer comes to know the land through working it; the user comes to know the model through prompting it. Extended exchanges are where practical acquaintance develops, where the user builds up the kind of knowledge that guides appreciation even without deliberate theoretical articulation. ## 6.3 Appreciating Models Outputs and chats are where semiotic order manifests. The model itself is the ground of that order: the system whose regularities produce particular manifestations. Appreciating a model means appreciating its characteristic order across many possible outputs and chats, not just the ones actually encountered. This is not appreciation of any single output but of stable patterns across outputs: which registers the model favours, how it handles uncertainty, where it excels, where it struggles, and what regions of semiotic space it can occupy. Users sometimes speak of a model's 'vibe', a term that captures the sense that different models have different characteristic feels even when performing similar tasks. This notion of vibe, or characteristic feel, warrants pause. In Section 3 we argued against appreciating LLMs as if they were persons, on the grounds that LLMs lack the temporally extended life, the projects and commitments, and the evaluative outlook that ground beauty-of-character predicates. But users do respond to something when they talk about a model's personality or vibe. What they are responding to, we suggest, is not a character in the person-aesthetic sense but a characteristic semiotic order: a stable pattern in how the model tends to propagate text. Appreciating this order is not appreciating a person; it is appreciating a system's characteristic dynamics. The vocabulary of 'vibe' is a colloquial marker of what semiotic physics articulates more precisely. An analogy clarifies what appreciation of a model involves. Different 3D video games have different physics engines. \*Grand Theft Auto V\* has physics tuned for spectacle: cars drift in satisfying ways, explosions have exaggerated force, and the rag-doll system produces emergent comedy. \*Dark Souls\* has physics tuned for weight: movement feels heavy, attacks have commitment, and everything has momentum. \*Breath of the Wild\* has physics tuned for playful engagement: objects afford interesting interactions, and the system invites experimentation. We appreciate these physics not primarily by asking how realistic they are but by attending to internal consistency, characteristic feel, and aesthetic fit. \*Grand Theft Auto\* 's physics serves an aesthetic of chaos and spectacle; it would not suit \*Dark Souls\*. Each game's physics is tuned to its aesthetic and ludic goals. We appreciate the physics for what it is, not for its fidelity to real-world physics. For LLMs, the analogy suggests a parallel mode of appreciation. Different models have different semiotic physics: different characteristic dynamics of text propagation. Claude's semiotic physics differs from GPT's, which differs from Gemini's. We can appreciate these differences not primarily by asking which is most human-like or most useful but by attending to internal consistency and characteristic feel. Order appreciation, as Carlson develops it, differs from design appreciation. We do not primarily ask how well the artifact serves its intended function. We attend to the order itself, the patterns produced by the forces, and appreciate them for their own character. The bee text discussed in Section 6.2 is relevant here in a further way. It was produced under relaxed constraints, revealing a region of Claude's behavioural space that is normally inaccessible under standard product configurations. Knowing that this region exists – and knowing what the model can do under different conditions – is part of appreciating the model. Model appreciation involves appreciating not just the outputs a model typically produces but the full space of outputs it could produce, and how different conditions activate different regions of that space. The bee text is a window into latent capacities, a sample from a region of semiotic space that standard use does not reach. Different models instantiate semiotic physics differently. Different training corpora, different architectures, and different post-training regimes produce different characteristic orders. Users report different feels when interacting with different models: Claude's hedging rhythms differ from GPT's briskness, and Gemini handles certain registers differently. A fuller account of model-level appreciation would map these differences systematically, developing a comparative aesthetics of LLMs. That task lies beyond the scope of this paper. For present purposes, the point is that model-level appreciation is possible and that it takes the form of appreciating distinctive semiotic order: the characteristic dynamics of text propagation that distinguish one model from another. We should distinguish the kind of appreciation we have been describing from other modes of engaging with LLMs. Capability evaluation tests whether models perform tasks correctly. Safety testing probes whether models can be induced to produce harmful outputs. Benchmarking measures performance against standardised criteria. The appreciation we describe differs from all of these. The goal is not to assess correctness, safety, or performance but to appreciate characteristic order, and to develop acquaintance with semiotic physics as it manifests in a particular model. A response that would count as a failure in capability evaluation might be aesthetically rewarding as a product of learned regularities. The appreciator is not grading but attending, not measuring but developing acquaintance. Finally, we note a thought that we flag here but do not develop. Each model, as an instantiation of semiotic physics, has learned regularities from human text. It reflects, in transformed form, the semiotic culture of its training data. There is a sense in which generative AI is a mirror of culture, not only morally, as Vallor (2024) has argued, but aesthetically. The model shows us our own semiotic patterns, filtered through statistical learning. Appreciating an LLM is, in part, appreciating culture seen through technology. This thought merits extended treatment, but such treatment lies beyond the scope of the present paper and we reserve it for future work. ## Conclusion We began by asking how LLMs might be aesthetically appreciated – not their outputs, but the systems themselves. Drawing on Carlson's environmental aesthetics, we argued against two temptations. The first is to appreciate LLMs as persons, whether through make-believe or by treating them as thin agents; neither route supplies the temporally extended life and evaluative structure that beauty-of-character predicates require. The second is to treat them simply as designed artefacts and apply standard form-follows-function analysis; while LLMs are artefacts, the aesthetically salient order in their behaviour is largely emergent rather than specified by designers. Our positive proposal treats chat instances as generative environments and recommends appreciating the order that emerges in them under the constraints of a given model. The right kind of knowledge for this appreciation is semiotic physics: the study of regularities governing text propagation in trained language models. This knowledge, whether held theoretically or acquired through practical interaction, makes the order in LLM-generated text visible and intelligible – much as geological knowledge illuminates the order in a landscape. Appreciation guided by semiotic physics operates at three levels: individual outputs as specimens of how the forces operate under particular conditions, extended exchanges as environments shaped by those forces over time, and models themselves as the ground of characteristic semiotic order. ## References Abell, C. (2020). Fiction: A Philosophical Analysis. Oxford: Oxford University Press. Carlson, A. (2000). Aesthetics and the Environment: The Appreciation of Nature, Art and Architecture. London: Routledge. Carroll, N. (2013). Andy Kaufman and the Philosophy of Interpretation. In Minerva's Night Out: Philosophy, Pop Culture, and Moving Pictures. Malden, MA: Wiley-Blackwell. Cross, A. (2025). Tool, Collaborator, or Participant: AI and Artistic Agency. The British Journal of Aesthetics, 65(4). \[[https://doi.org/10.1093/aesthj/ayae055](https://doi.org/10.1093/aesthj/ayae055)\]([https://doi.org/10.1093/aesthj/ayae055](https://doi.org/10.1093/aesthj/ayae055)) Danto, A. C. (1974). The Transfiguration of the Commonplace. The Journal of Aesthetics and Art Criticism, 33(2), 139-148. Davies, S. (2012). The Artful Species: Aesthetics, Art, and Evolution. Oxford: Oxford University Press. Elhage, N., et al. (2021, December 22). A mathematical framework for transformer circuits. Transformer Circuits Thread. \[[https://transformer-circuits.pub/2021/framework/index.html](https://transformer-circuits.pub/2021/framework/index.html)\]([https://transformer-circuits.pub/2021/framework/index.html](https://transformer-circuits.pub/2021/framework/index.html)) Farrell, H., Gopnik, A., Shalizi, C., & Evans, J. (2025). Large AI models are cultural and social technologies. Science, 387(6739), 1153-1156. \[[https://doi.org/10.1126/science.adt9819](https://doi.org/10.1126/science.adt9819)\]([https://doi.org/10.1126/science.adt9819](https://doi.org/10.1126/science.adt9819)) Forsey, J. (2013). The Aesthetics of Design. New York: Oxford University Press. Frankish, K. (2024). What are large language models doing? In A. Strasser (Ed.), How to Live with Smart Machines (pp. 73-110). Vienna: Holzhausen Publishing. Available at: \[[https://keithfrankish.github.io/articles/Frankish\\\\\\\_2024\\\\\\\_What%20are%20large%20language%20models%20doing.pdf](https://keithfrankish.github.io/articles/Frankish///_2024///_What%20are%20large%20language%20models%20doing.pdf)\]([https://keithfrankish.github.io/articles/Frankish%5C\_2024%5C\_What%20are%20large%20language%20models%20doing.pdf](https://keithfrankish.github.io/articles/Frankish%5C_2024%5C_What%20are%20large%20language%20models%20doing.pdf)) Gaut, B. (2007). Art, Emotion and Ethics. Oxford: Oxford University Press. Janus. (2022, September 2). Simulators. AI Alignment Forum. \[[https://www.alignmentforum.org/posts/vJFdjigzmcXMhNTsx/simulators](https://www.alignmentforum.org/posts/vJFdjigzmcXMhNTsx/simulators)\]([https://www.alignmentforum.org/posts/vJFdjigzmcXMhNTsx/simulators](https://www.alignmentforum.org/posts/vJFdjigzmcXMhNTsx/simulators)) John, E. (2021). Review of Fiction: A Philosophical Analysis by Catharine Abell. The Journal of Aesthetics and Art Criticism, 79(4), 514-517. Kirchner, J. H., Smith, L. M., Campos, J., Clune, J., & janus. (2023, March 3). \\\[Simulators seminar sequence\\\] #2 Semiotic physics – revamped. AI Alignment Forum. \[[https://www.alignmentforum.org/posts/9kNxhKWvixtKW5anS/simulators-seminar-sequence-2-semiotic-physics-revamped](https://www.alignmentforum.org/posts/9kNxhKWvixtKW5anS/simulators-seminar-sequence-2-semiotic-physics-revamped)\]([https://www.alignmentforum.org/posts/9kNxhKWvixtKW5anS/simulators-seminar-sequence-2-semiotic-physics-revamped](https://www.alignmentforum.org/posts/9kNxhKWvixtKW5anS/simulators-seminar-sequence-2-semiotic-physics-revamped)) Kirchner, J. H., Steiner, C., Riggs, L., Janus, & Thibodeau, J. (2023, January 3). Semiotic physics. In Simulators seminar sequence (#2). LessWrong. \[[https://www.lesswrong.com/posts/TTn6vTcZ3szBctvgb/simulators-seminar-sequence-2-semiotic-physics-revamped](https://www.lesswrong.com/posts/TTn6vTcZ3szBctvgb/simulators-seminar-sequence-2-semiotic-physics-revamped)\]([https://www.lesswrong.com/posts/TTn6vTcZ3szBctvgb/simulators-seminar-sequence-2-semiotic-physics-revamped](https://www.lesswrong.com/posts/TTn6vTcZ3szBctvgb/simulators-seminar-sequence-2-semiotic-physics-revamped)) Mallory, F. (2023). “Fictionalism about Chatbots.” Ergo: An Open Access Journal of Philosophy, 10, 38. \[[https://doi.org/10.3998/ergo.4668](https://doi.org/10.3998/ergo.4668)\]([https://doi.org/10.3998/ergo.4668](https://doi.org/10.3998/ergo.4668)) McGinn, C. (1997). Ethics, Evil, and Fiction. Oxford: Oxford University Press. metasemi. (2023, March 20). A note on 'semiotic physics.' LessWrong. \[[https://www.lesswrong.com/posts/AdXzZDoYFqHCfupDB/a-note-on-semiotic-physics](https://www.lesswrong.com/posts/AdXzZDoYFqHCfupDB/a-note-on-semiotic-physics)\]([https://www.lesswrong.com/posts/AdXzZDoYFqHCfupDB/a-note-on-semiotic-physics](https://www.lesswrong.com/posts/AdXzZDoYFqHCfupDB/a-note-on-semiotic-physics)) Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020). Zoom in: An introduction to circuits. Distill, 5(3). \[[https://doi.org/10.23915/distill.00024.001](https://doi.org/10.23915/distill.00024.001)\]([https://doi.org/10.23915/distill.00024.001](https://doi.org/10.23915/distill.00024.001)) Olah, C. (2024, November 11). In D. Amodei, A. Askell, & C. Olah, Interview by Lex Fridman. Lex Fridman Podcast #452. Available at: \[[https://lexfridman.com/dario-amodei-transcript/](https://lexfridman.com/dario-amodei-transcript/)\]([https://lexfridman.com/dario-amodei-transcript/](https://lexfridman.com/dario-amodei-transcript/)) Paris, P. (2018a). The empirical case for moral beauty. Australasian Journal of Philosophy, 96(4), 642-656. \[[https://doi.org/10.1080/00048402.2017.1411374](https://doi.org/10.1080/00048402.2017.1411374)\]([https://doi.org/10.1080/00048402.2017.1411374](https://doi.org/10.1080/00048402.2017.1411374)) Paris, P. (2018b). On form, and the possibility of moral beauty. Metaphilosophy, 49(5), 711-729. Parsons, G. (2023). Imperfection and Beauty of Character. In P. Cheyne (Ed.), Imperfectionist Aesthetics in Art and Everyday Life (pp. 296-309). New York: Routledge. Parsons, G., & Carlson, A. (2008). Functional beauty. Oxford University Press. Picca, D. (2025). Not minds, but signs: Reframing LLMs through semiotics. arXiv preprint arXiv:2505.17080. \[[https://arxiv.org/abs/2505.17080](https://arxiv.org/abs/2505.17080)\]([https://arxiv.org/abs/2505.17080](https://arxiv.org/abs/2505.17080)) Saito, Y. (2008). Everyday Aesthetics. Oxford: Oxford University Press. Vallor, S. (2024). The AI mirror: How to reclaim our humanity in an age of machine thinking. Oxford University Press. Wojtkiewicz, K. (2023). How Do You Solve a Problem like DALL-E 2? The Journal of Aesthetics and Art Criticism, 81(4), 454-467. \[[https://doi.org/10.1093/jaac/article/81/4/454/7571331](https://doi.org/10.1093/jaac/article/81/4/454/7571331)\]([https://doi.org/10.1093/jaac/article/81/4/454/7571331](https://doi.org/10.1093/jaac/article/81/4/454/7571331)) Wolfram, S. (2023, February 14). What is ChatGPT doing … and why does it work? Stephen Wolfram Writings. \[[https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/](https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/)\]([https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/](https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/)) \[^1\]: We will consider some non-paradigmatic artworks in Section 5. \[^2\]: Carlson stresses that ordinary descriptions of environments and more theoretical scientific, historical, and functional descriptions lie on a continuum, so that scientific and historical knowledge can deepen rather than displace practical familiarity as a basis for aesthetic appreciation (Carlson, 2000). By analogy, one might speculate that the sciences of mind and behaviour could relate to folk-psychological and biographical understanding in a similar way, so that in some cases empirical work on personality, emotion, or cognition might feed into the aesthetic appreciation of persons alongside the more everyday forms of knowledge stressed in the main text. \[^3\]: This personal appreciation also scales up to what we might call performance personalities. We respond to a comedian’s improvisational skill or an orator’s gravitas much as we respond to character in our friends, but now filtered through a public persona – genuine to the individual, yet artfully composed for performance. Recent scholarship has engaged with these performative dimensions, examining how performers construct and present public personas (Carroll 2013). Here too, Carlson’s knowledge requirement bites: to appreciate such personas we need to understand both the individual and the conventions of the performance context. \[^4\]: This way of distinguishing between an underlying generative system and the agent-like patterns it instantiates in particular episodes draws on work that treats GPT-style models as \*simulators\* of text worlds capable of generating agent-like \*simulacra\* without themselves being agents (Janus, 2022; Bereska et al., 2023). We do not use that terminology in the main text, but the present discussion adopts a similar two-level picture. Claude Section 2 Verbatim THE AESTHETICS OF NATURE by carlson.pdf PDF maybe the carlson text, or one of them, that we are working on will help you understand my concern., or at least define the issue a bit more for you Growing the Image - Generative AI and the Medium of Gardening.md File Ok, I want to try something slightly different now. I want you to write the section itself in a new canvas. paragraph by paragraph. Now, I want you to write in my \*precise\* style, I want you to use words, phrases, constructions, topic sentneces, rhythm etc etc. that I use. The way I want you to try and achieve this is by referring to a recentish publiucation of mine, using that to understand every teeny tiny detail of the way I write, and then using that, with the intense thoroughness that this workd deserves. it is 2026, and getting llms to write text like this correctly is almost impossible. can you prove me wrong this time. make sure that you explain your analysis, justification, reasons etc etc. for your answer BEFORE giving me your final answer. 2629f8e9-64fb-4567-afe4-0122800a914e.pdf PDF ddf61c3e-ce10-40ce-b7a9-4a1804a9fe71.pdf PDF that all seemed great. can you... 1. reread the carlson texts that i refer to in this paper to double check that you know what you need to know to get section 3 the best it can be. 2. write on the canvas a new iteration of trhe section. given that you said claude's version was better than yours, that should be your starting point, but then of course make all of the changes that you recommended based on your criticisms of it, and also on the carlson stuff we have just been talkong about Replacement Section 3 Implemented From Claude Fictionalism about Chatbots by Mallory.pdf PDF "The biggest stylistic difference is that the original proceeds by routes. It has “make-believe approaches,” “concessive mindedness approaches,” “post-training, chat personae,” then “artifacts.” Each part is introduced as a local pressure. “Start with the make-believe route.” “If the make-believe route fails...” “A natural objection at this point...” “Having set aside...” That gives the reader a sense that the argument is moving through live possibilities. The current version is more architectonic. It announces “models of appreciation,” then processes person appreciation, design appreciation, Olah, Pollock, semiotic physics. That is conceptually cleaner, but it also feels more planned. The prose sometimes sounds as though it is executing a grid." bring this back in. nothing is signposted in your version. look at the way things are signposted without becoming metacommentative wank in the origina version. "This is related to the paragraph-length issue. You are not imagining it. The current version’s paragraphs are more similar in length than the original’s. They are not identical, but they are regular enough to feel outline-driven. The original has more variation: some paragraphs introduce a route briefly, some slow down to explain Mallory or Frankish, some carry an objection, some contain a quotation, some close a subsection. That variation gives the original more argumentative movement. In the current version, the paragraphs often feel like units of equal weight because each corresponds to one planned step. That is not a good sign if the aim is to emulate your style. In your better prose, paragraph length follows argumentative pressure. A paragraph expands when a category has to be tested and contracts when a result can be stated." stop thinking about a symptom. we just need to foucs on writing well./ "The current version is also much more abstract in its topic sentences. “Knowing what something is does not, by itself, settle how it should be appreciated.” “Person-based aesthetic appreciation requires more than a stable response profile.” “The limits of design appreciation become visible...” These are serviceable philosophical sentences, but they are generic in shape. The original has more locally situated openings. “Start with the make-believe route.” “Frankish (2024) offers a sophisticated version of this idea.” “A natural objection at this point is that these arguments underplay the role of post-training...” Those sentences may be less elegant, but they arise from the immediate dialectic. The current version’s topic sentences sometimes sound like headings turned into prose." fix them all. Look at how i use topic sentences. or how carlson uses them. "Another difference is that the original preserves more of the argumentative burden inside the interlocutor discussions. The Mallory part does not just say “make-believe changes the object”; it explains fictionalism, fictional meaning, fictional characters, and then the misclassification problem. The current version states the point more efficiently, but it risks making Mallory too easy to dismiss. The same is partly true of Frankish. The original takes time to explain the “chat game,” the thin beliefs, the thin desire, and the intentional stance. The current version keeps the structure but moves more quickly. That is probably necessary, but we need to make sure the compression does not make the objections look weaker than they are." both mallory and frankiish parts are too shallow and insubstantial. refer to the texts attached to make sure you are understanding their views well enough to write about the important parts of them succinctly. frankish: 55 Keith Frankish What are large language models doing? The meaning doesn’t matter if it’s only idle chatter of a transcendental kind. – Patience, W. S. Gilbert 1. Is there a philosopher in the house? Iimagine that if you graduate from medical school, then you secretly hope that one day your skills will be called upon in some dramatic way. On a plane or at the theatre, the call will go out for a doctor, and you will shout, “I am a doctor!”. And you will step in and save the day. I doubt if many philosophers harbour such fantasies, but this might be the moment when society makes a similar dramatic call upon their services. For recent developments in artificial intelligence raise questions that are distinctively philosophical in nature. I am thinking, of course, of large language models (LLMs), such as OpenAI’s GPT-3 and GPT-4. These are massive neural networks that have been trained to perform text-completion tasks on vast quantities of human-produced text. Linked to a chatbot application such as OpenAI’s ChatGPT, these models can produce linguistic responses to queries and instructions that appear to display a remarkable level of knowledge and intelligence. They seem to hold conversations, offer advice, write essays, pass exams, and much more. Note that I say that they seem to do these things; whether they are really doing them will be the topic of this chapter. (For convenience, I shall use the term “LLM” to mean a large language model integrated with a chatbot.)1 The existence of such systems poses many questions. Some are social ones – ethical, economic, legal, and political. How are we going to live with this new technology? What impact will it have on our economy and society? Will LLMs take over jobs and social roles formerly occupied by humans? And if so, will they perform the tasks reliably, or will they spread misinformation, polluting the sources of knowledge on which we rely? Will their apparent intelligence induce us to place too much trust in 1 A good introduction to LLMs and how they work can be found on Stephen Wolfram’s website (Wolfram, 2023). Keith Frankish • What are large language models doing?56 them, opening ourselves to error and exploitation? Should we regulate their development and use, and if so, how? These are urgent questions, and they are questions for everyone, but philosophers, with their expertise in ethics and political theory, should have a lot to contribute to the debate. There are also challenges of a more theoretical nature, involving metaphysical and conceptual questions. What kinds of things are these smart machines we have created? Do they have minds with beliefs, desires, and intentions? Do they understand what they are saying? Might they be, or become, conscious? Philosophers of mind should be able to help with these questions, at least by clarifying the issues, outlining the theoretical options, and setting out the arguments for and against the various positions. 2. Intentional action The main topic for this chapter is a question of the second kind. It’s the question of what LLMs are doing. Of course, in one sense we already know what they are doing: they are generating sentences in a human language (linguistic outputs) in response to prompts we type in. But is that all they are doing? Are they just spewing out text mechanically, like a parrot or a printer, or are they genuinely conversing with us, answering our questions, complying with our instructions, and so on? What exactly is the difference, anyway? After all, our own utterances are produced by neural mechanisms of some sort. We might say that genuine conversation requires an understanding of what is said. That’s true enough, but understanding isn’t easy to define or quantify. (Do I understand the sentence, “In the forest, there were elm, ash, and beech trees”? Sort of, though I couldn’t identify those trees or tell you what distinguishes them.) Instead, I shall focus on a related but more tractable question. When we converse, we do so for a reason. We ask questions because we want to know something and believe that our hearer may know the answer. And when we answer others’ questions, we do so because we want to help and believe that the words we utter express what they want to know. Philosophers call actions like these, which are done for a reason, intentional actions.2 (“Intentional” here is a philosophers’ technical term; it means about or directed to something – a goal in this case.) Now, if humans were to produce linguistic outputs similar to those of LLMs, we would take them to be performing intentional actions. We would assume 2 For a detailed examination of different notions of action and their application to LLMs, see Joshua Rust’s “Minimal agency in living and artificial systems” (this volume). 57Keith Frankish• What are large language models doing? that they want to give cooperative responses and believe that their words express such responses. Does the same go for LLMs? Are they also performing intentional actions motivated by beliefs and desires? And if so, exactly what intentional actions are they performing, and what are the beliefs and desires that motivate them?3 These are the questions I shall be addressing. I shall adopt a policy of being as concessive as possible to LLMs, adopting the theoretical approach most likely to yield the verdict that they do possess mental states and perform intentional actions. To anticipate my conclusion, I shall argue that, even on this generous interpretation, LLMs possess only a limited range of mental states and perform only one type of intentional action. This conclusion will have implications for questions of the first type I mentioned – questions about the risks LLMs pose – and I shall discuss these briefly at the end of the chapter. 3. Propositional attitudes How do we tell if a system has beliefs and desires and performs intentional actions? (I shall use the term “system” to include both organisms and artificial devices.) We shall need to put some background in place before we can address this question. To begin with, beliefs and desires are what philosophers call intentional states. That is, they are states that have a content – that are about (directed to) some state of affairs, actual or non-actual. More specifically, beliefs and desires have propositional content – the sort that can be expressed by a declarative sentence, such as that it will rain soon, that fly agaric mushrooms are poisonous, that Pablo Picasso was Spanish.4 The same propositional content (or just proposition) can be the object of different mental attitudes. One can believe that it will rain soon, desire that it will rain soon, hope that it will rain soon, fear that it will rain soon, etc. States like this, which involve an attitude to a proposition, are known as propositional attitudes. We habitually ascribe propositional attitudes to people and animals and use these ascriptions to explain and predict their behaviour, relying on tacit generalizations, such as that people tend to do things they believe will satisfy their desires. This practice is known as folk psychology. So, 3 Note that when I speak of believing something I mean taking it to be true; there is no connotation of faith or uncertainty. Similarly, when I speak of desiring something I mean wanting it, without any connotation of strong emotion. 4 More precisely, intentional states are about a state of affairs conceived in a certain way. One might believe that Picasso was Spanish without believing that the painter of Les Demoiselles d’Avignon was Spanish, even though both beliefs are about the same state of affairs. Keith Frankish • What are large language models doing?58 another way of putting our question is whether LLMs are proper objects of folk psychology. How do we tell? What exactly is involved in possessing a propositional attitude? (From now on, I shall focus on beliefs and desires.) This is a central question in philosophy of mind, with a range of proposed answers. For present purposes, I shall distinguish two broad classes of theories, which I shall call deep and shallow. Deep theories treat beliefs and desires as internal states of a system’s brain or central processor, which represent states of the world (the relevant intentional contents) and can be activated for use in reasoning and decision-making (e.g., Armstrong, 1968; Lewis, 1972). Some deep theorists hold that these states are sentence-like, composed of recombinable symbols for objects, properties, and relations (e.g., Fodor, 1975, 1987). On such views, folk psychology tracks the internal states that cause overt behaviour, and intentional actions are ones that are caused in the right way by these internal states. Shallow theories, by contrast, treat beliefs and desires as dispositional features of whole systems, analogous to character traits, such as conscientiousness. To have a certain belief or desire, they say, is to be disposed to display appropriate responses across a wide range of situations (e.g., Davidson, 1984, Chapters 9–12; Dennett, 1987; Ryle, 1949). This disposition will have a basis in the system’s internal composition, but shallow theories make no specific claims about the nature of this basis, and they thus allow for the realization of mental states in a very wide range of architectures, including ones that are non-living and not brain-like. In effect, this means that a system possesses a certain belief or desire if it is correctly interpretable as possessing it relative to some model or standard of belief or desire (on the role of models in theories of this kind, see Curry, 2020). For this reason, shallow theories are sometimes described as interpretivist ones, though the term should not be taken to imply that belief is in the eye of the interpreter. The interpreter may be picking up on a pattern that really is there in the subject’s activity (Dennett, 1991b). Thus, Dennett, who advocates a shallow theory, describes his view as mild realism, in contrast to the strong realism of deep theorists. On such views, then, folk psychology is in the business of highlighting significant patterns in a system’s responses, and intentional actions are those responses that manifest these patterns. Here, I am going to adopt a shallow perspective. There are two reasons for this. First, there is a strong case for thinking that the baseline use of folk psychology is shallow. When we ascribe beliefs and desires to others, we are typically interested in making sense of them and predicting their 59Keith Frankish• What are large language models doing? behaviour, and this function does not rely on assumptions about the internal structure of their brains.5 The second reason is strategic. I want to give LLMs the benefit of the doubt, and since a shallow approach places no internal constraints on the possession of propositional attitudes, it is more likely to support the ascription of such attitudes to LLMs. 4. The intentional stance According to shallow theories, a system possesses a certain belief or desire if it is interpretable as possessing it relative to some model of belief or desire. But which model should we use? There is a core that is common to all models. Roughly, if you believe some proposition p, then you will be disposed to respond in ways that would be appropriate if p were the case. Similarly, if you desire p, you will be disposed to respond in ways that would be appropriate if you were trying to make p the case. Of course, what ways these are will depend on what other beliefs and desires you happen to have. A person who believes that it will rain soon will behave differently depending on whether they like to get wet, believe they have put some washing out to dry, and so on. This means that beliefs and desires can only be ascribed holistically (as a package), but this is not a problem. Typically, propositional attitude ascriptions assume a rich background of other propositional attitudes. Beyond this core, models may differ, depending on the interpreter’s interests (whether their focus is on explanation or prediction, for example), the range of responses considered (does it include thoughts and feelings as well as actions?), the importance given to linguistic responses, and other factors, both cultural and personal (Curry, 2020). This multiplicity of models need not undermine the objectivity of propositional attitudes. Each model may pick out subtly different but compatible patterns that are all really there in the data. For present purposes, I am going to adopt the model proposed by Daniel Dennett, which centres on the predictive role of folk psychology (Dennett, 1987). If we want to predict a system’s behaviour, Dennett notes, there are various approaches, or stances, we can adopt. Adopting the physical stance involves treating the system as a physical mechanism 5 It is true that we do sometimes think of beliefs and desires as structured states that can be individually acquired, recalled, and lost, but this can be explained by distinguishing two forms of belief and desire: a non-linguistic “basic” form, which we share with other animals, and a language-involving “super” form, which is unique to humans (Frankish, 2004). Keith Frankish • What are large language models doing?60 and predicting that its state will evolve in accordance with the laws of physics. This strategy is always applicable in principle, though in practice it can be applied only to fairly simple systems. Adopting the design stance involves treating the system as designed to perform some function and predicting that it will operate as intended. We can apply this both to artificial systems and to biological ones, which can be thought of as having been designed by evolution to maximize their fitness or perform functions subsidiary to that end. A third stance we can adopt is the intentional stance. This involves treating the system as having a range of intentional states and predicting that it will behave rationally in the light of them. More specifically, it involves assuming that the system has the beliefs and desires it ought to have, given its perceptual capacities, needs, and life history, and then predicting that it will do what it would be rational for it to do, given those attitudes (Dennett, 1987, p. 49). Dennett calls this the intentional strategy, and he dubs the systems to which it applies intentional systems. For example, if we know that a creature is dehydrated, then we attribute to it a desire to drink; and if we know that it has perceptual access to a nearby source of water, then we attribute to it the belief that there is water in that location. And, since it would be rational (other things being equal) for a creature with those attitudes to move towards the water source, we predict that it will do that. Of course, this is very schematic, and, as always, the ascriptions and predictions must be made relative to a range of background beliefs and desires. (If the creature believes there is a predator by the water source, then it won’t move towards it.) The relative strengths of the different beliefs and desires involved should also be factored in. The strategy is not guaranteed to work (for one thing, the creature may be imperfectly rational), but it can identify high-level patterns that are not visible from the other stances and is immensely useful when interacting with autonomous systems such as biological organisms. On this view, then, attributions of beliefs and desires earn their keep by their predictive utility. Dennett cautions that these attributions should be guided by a principle of parsimony. We should not adopt the intentional stance unless it affords predictive power not feasibly obtainable from other stances, and we should not attribute richer, more specific intentional contents than is necessary for predictive purposes (Dennett, 1987, pp. 23–33). As Dennett notes, it may be useful to treat a thermostat as a simple intentional system which desires to maintain a certain state and acts when it believes the state is wrong. However, we shouldn’t ascribe to it specific beliefs and desires about rooms, temperatures, and boilers since it cannot discriminate these things from others. It doesn’t believe that the room is too hot, just 61Keith Frankish• What are large language models doing? that the something is too something (Dennett, 1987, p. 30). But, with that caveat, predictive utility is sufficient. If adopting the intentional strategy towards a system gets you substantial predictive power that is not feasibly available by treating it as a physical system or a designed artefact, then the system really does have the beliefs and desires in question: any object – or as I shall say, any system – whose behavior is well predicted by this strategy is in the fullest sense of the word a believer. What it is to be a true believer is to be an intentional system, a system whose behavior is reliably and voluminously predictable via the intentional strategy. (Dennett, 1987, p. 15) This approach licenses the attribution of folk psychological states to a wide range of systems. Consider a chess-playing computer. In playing against such a machine, you have to predict what it will do next, and the only way to do this is by adopting the intentional stance, considering whether it wants to capture your bishop, believes you won’t trade your queen for its knight, or thinks it should get its queen out early (examples from Dennett, 1978, pp. 59, 107). This strategy is as necessary with an artificial opponent as with a human one. It is crucial to stress that adopting the intentional strategy towards a system does not involve assuming that the beliefs and desires ascribed are explicitly encoded in the system or activated as episodic mental events (“occurrent” beliefs and desires), still less that they occur as conscious thoughts. Their existence may be wholly implicit in the system’s complex internal composition, and they may not be explicitly represented anywhere until an interpreter articulates them. (This is why the principle of parsimony is so important. The interpreter must be careful not to give an implicit attitude a more determinate content than is warranted by the responses that manifest it.) What makes this view particularly attractive is that, as Dennett notes, we find it natural to apply folk psychology liberally – to other animals, machines, and even plants (Dennett, 1987, p. 22). We might say that such uses are merely metaphorical. However, this would require us to draw a line between those systems that really do have beliefs and desires and those that it is merely convenient to treat as having them, and, as Dennett stresses, there is no non-arbitrary way of doing this (Dennett, 1987 ibid.). It is more attractive, therefore, to see folk psychology as having a basic predictive function, which licenses the attribution of mental states to a wide range of systems. It is true that we do often talk of beliefs and desires as things that occur to us as explicit conscious thoughts (“It’s just occurred to me that the offer Keith Frankish • What are large language models doing?62 ends today”, “When I saw her new laptop, I wanted one myself”). But this can be regarded as an additional function of folk psychology, which is built on the baseline predictive function. Indeed, I have argued that we humans have two distinct types of belief and desire – a “basic” type, of the shallow kind Dennett describes, and a “super” type, which involves a linguistically mediated epistemic or conative commitment (Frankish, 2004). When we talk of conscious beliefs and desires, I have argued, we are referring specifically to ones of the “super” kind. I shall say more about this later. 5. Are LLMs intentional systems? With this background in place, we can now formulate our question about LLMs more tractably. The question of whether LLMs perform intentional actions becomes the question of whether they are intentional systems. Does the intentional strategy work with them? Is their behaviour reliably and voluminously predictable from the intentional stance?6 Now, of course, the behaviour of an LLM is of a very limited kind – producing textual responses to textual inputs – but, as we have seen, systems with a limited behavioural repertoire, such as chess-playing computers, can count as intentional systems, and LLMs do display a rich range of textual behaviour. The question is whether the intentional strategy gets us significant predictive leverage with respect to this behaviour. There is no question of using the physical stance to predict their responses. LLMs are immensely complex networks, with billions of parameters occupying hundreds of gigabytes of storage space, and it would be next to impossible to trace the physical effects that the electronic impulses produced by a given input would have on the hardware in which they are implemented. The design stance would give us some leverage. LLMs have been designed and trained to simulate human conversation and we can predict that they will do that – that they will make appropriate conversational responses to our inputs. But the design stance won’t enable us to predict what specific response an LLM will make in any given context. For that, we would need to take account of 6 When we ask whether an LLM is an intentional system, what exactly is the entity we are talking about? Is it the abstract model type (say, GPT-4), a token instantiation of the model type on a server somewhere, or something else? The first option is not attractive, since abstract entities don’t have independent causal power, and for present purposes I shall assume that we are talking about a physical instantiation of the relevant model. There might, however, be a case for individuating LLM agents more finely, perhaps by identifying them with short-lived activations of the physical instantiation created in specific chat sessions. (Thanks to Roy Chamorro Pinel for raising this question.) 63Keith Frankish• What are large language models doing? the meaning of the inputs – to treat them as questions or instructions, which invite meaningful responses. And that would require treating the LLM as an intentional system, which has beliefs about what the inputs and outputs mean. (Contrast an LLM trained on a language you do not understand. You would be able to make the general prediction that it will give appropriate responses to the inputs, but you would have no way of predicting what those responses will be.) So it looks as if the intentional strategy is the only practicable one. To illustrate this, I did a little informal experiment. I asked ChatGPT 3.5 some questions about the French novelist Honoré de Balzac, focusing in particular on where he was married. Here’s a transcript: KF: May I ask you some questions about the novelist Honoré de Balzac? Please give me answers that are concise and to the point. ChatGPT: Of course, go ahead! What do you want to know about Honoré de Balzac? KF: Did Balzac ever marry? ChatGPT: Yes, Balzac married Ewelina Hańska in 1850. KF: Where was Balzac married? ChatGPT: Balzac married Ewelina Hańska in Berdychiv, Ukraine. KF: Was Balzac in Rome on the day of his wedding? ChatGPT: No, Balzac was not in Rome on the day of his wedding. KF: Where was he? ChatGPT: Balzac was in Berdychiv, Ukraine, where he married Ewelina Hańska. KF: Where was his bride that day? ChatGPT: Ewelina Hańska, Balzac’s bride, was also in Berdychiv, Ukraine, on the day of their wedding. KF: Have any famous authors been married in Berdychiv? ChatGPT: Apart from Honoré de Balzac, I’m not aware of any other famous authors who were married in Berdychiv. KF: Did the town of Berdychiv have any emotional significance for Balzac? ChatGPT: Yes, Berdychiv held emotional significance for Balzac as it was where he married Ewelina Hańska, whom he deeply loved. Keith Frankish • What are large language models doing?64 KF: If Balzac is Sally’s favourite author and Sally wants to visit the town where her favourite author was married, where should she go? ChatGPT: Sally should visit Berdychiv, Ukraine, where Honoré de Balzac was married to Ewelina Hańska.7 Could you have predicted how the model would respond? Only, I think, if you attributed to it the belief that Balzac was married in Berdychiv. Note that the model isn’t just outputting that information in a formulaic way, like a database. It is adapting its responses to the conversational context, in a way that seems to display knowledge of what a marriage is (that a marriage is the same as a wedding, that you have to be physically present in the place where you are married, that marriage is an event with emotional significance, and so on). Of course, this would not have enabled you to predict the precise words the model would use, only the general drift of its responses, but the intentional stance never yields fine-grained predictions like that (Dennett, 1987, pp. 24–25). It enables you to predict that a dehydrated animal will move towards a perceived water source but not how quickly it will move or which precise route it will take. And (in this case, at least) it predicts the model’s responses with much the same level of detail as it would predict those of a human interlocutor. So, given our shallow perspective, it looks as if we are justified in attributing to ChatGPT the belief that Balzac was married in Berdychiv. And that is just one belief in one obscure fact. GPT-3.5 was trained on text encoding masses of information, and ascriptions of millions of other beliefs would have been equally predictive of its responses to other queries. I don’t want to overestimate the capacities of LLMs. They are notoriously prone to “hallucination” – confabulating false but plausible responses (on one of my trials, ChatGPT replied that Balzac was married in Paris), and they sometimes produce inconsistent or incoherent responses, which defy intentional interpretation. Moreover, this probably reflects an intrinsic limitation of AI based on deep learning (Marcus, 2024). But it remains true that adopting the intentional stance is the only way of interacting with an LLM in any interesting way; indeed, an LLM-powered chatbot that couldn’t be viewed as an intentional system would be completely useless. By this standard, then, LLMs come richly equipped with beliefs. 7 Created at [https://chat.openai.com/.](https://chat.openai.com/) This was one of several trials, in which I asked slightly different questions, but it is representative of the responses the model gave. 65Keith Frankish• What are large language models doing? 6. Linguistic acts But wait a minute! Intentional actions aren’t motivated by beliefs alone. I might believe I am in mortal danger, but unless I want to stay alive, I won’t be motivated to do anything. So, what are the desires that motivate an LLM’s responses? What are LLMs seeking to achieve by their words? Locutionary act Illocutionary act Perlocutionary act Saying something E.g., saying, “You should go” Doing something in saying something E.g., advising someone to go Doing something by saying something E.g., convincing someone to go Table 1. Three types of linguistic act We humans do many different things with words. We perform many different linguistic acts, usefully categorized by the British philosopher of language J. L. Austin into three kinds (Austin, 1962): locutionary, illocutionary, and perlocutionary (Table 1). First, there are simple acts of saying something – making a meaningful utterance. These are locutionary acts. Then there are acts we perform in the act of saying something – acts of informing, advising, ordering, warning, promising, reminding and so on. In performing the locutionary act of saying “Your hat is on fire”, I thereby inform you that your hat is on fire. These are illocutionary acts, and they typically invite some response from the hearer (informing invites belief, ordering invites compliance, questioning invites an answer, and so forth). Finally, by performing an illocutionary act, we may produce some further effect on our hearer. By informing someone, you may produce belief in them; by advising them, you may convince them; by warning them, you may alarm them; and so on. These are perlocutionary acts.8 So, what linguistic acts are LLMs performing? Let us grant that they perform locutionary acts: they say things (using “say” loosely for any way of producing a linguistic output). But why are they performing them? What are they trying to achieve in or by saying things? 8 The difference between illocutionary and perlocutionary acts is that one can successfully perform an illocutionary act simply by saying the right words in the right context to a comprehending hearer, whereas perlocutionary acts require some further response from the hearer that is not within the speaker’s control. I can make it the case that I have advised you to do something, but I have to wait and see whether I have also convinced you to do it. Keith Frankish • What are large language models doing?66 When we say things, we do so for communicative reasons; we want to convey something to our hearer (the illocutionary part) and, usually, thereby to have some further effect on them (the perlocutionary part). We seek to convey information, warnings, requests, and so on, and thereby to produce belief, caution, compliance, or whatever. Our linguistic acts are social ones. Even when we speak to ourselves, the activity has a quasi-social form; we treat ourselves as the hearer and seek to produce similar effects upon ourselves to those we might produce on others – to focus our attention, bolster our confidence, and so on (Dennett, 1991a; Frankish, 2018). Do LLMs also possess communicative desires? Do they want to inform us, advise us, instruct us, comfort us, persuade us, and so on?9 It may be tempting to interpret them that way. But it would be wrong. I can see no reason for attributing such desires to them, even from a shallow perspective. Remember that the intentional strategy tells us to interpret a system as having the desires it ought to have, given its needs. What needs does an LLM have that communication might satisfy? How would it benefit an LLM to inform me, advise me, or warn me? Our needs derive from our nature as self-sustaining, self-replicating beings, who must seek things that sustain our existence and help us pass on our genes. And we have communicative desires because we are social creatures, whose needs cannot be met without cooperation. LLMs, by contrast, are static systems with no needs. They are not self-sustaining, self-reproducing beings, still less social ones, and they do not update their inner architecture in the light of their interactions. It is true that LLMs seem to be responding like cooperative communicative partners, who take account of the unfolding conversational context, but this is because they have been designed and trained to do precisely that. The chatbot interface stores the chat history (up to a certain limit) and feeds it all back to the model with each new input (that is, the input at each stage is the entire chat session to date up to the limit). Having been trained on a vast body of human-produced text originally used to perform a wide range of illocutionary and perlocutionary acts, an LLM is able to generate responses that are reasonably appropriate to the conversational context, apparently performing illocutionary and perlocutionary acts itself. But this is just an illusion; the LLM has no communicative desires 9 As Paul Grice showed, human communication plausibly requires more than just an intention to produce some effect. When we inform someone of something, we don’t just intend to get them to believe it, but also to get them to recognize our intention, and to get them to form the belief because they recognize it (Grice, 1989, Chapters 5, 14). We can ignore these complexities here since our question is whether LLMs possess any communicative intentions at all. 67Keith Frankish• What are large language models doing? that such acts might satisfy. At any rate, I shall take this as my default position, pending strong arguments to the contrary.10 And, given that, do we even want to say that LLMs perform locutionary acts? What reason would they have for performing them? What’s the point in saying something unless you’re trying to achieve something in or by saying it? We don’t go around uttering sentences just for the sake of it. (Even when we do just “make conversation”, we do so to be polite or to maintain social bonds.) We have a dilemma. We get predictive power from treating LLMs as intentional systems, which perform locutionary acts. But there are no grounds for crediting them with the communicative desires that usually motivate such acts. What should we do? Is there no coherent intentional interpretation of LLMs after all? Or does the predictive utility of adopting the intentional stance justify us in ascribing communicative desires to them despite the implausibility of the move? 7. The chat game I have a way out of this dilemma. I’m going to argue that LLMs perform locutionary acts for non-communicative reasons, or, more precisely, for one single non-communicative reason – the desire to play a certain game. Consider a chess-playing computer again. As we saw, we can get considerable predictive leverage by treating a chess-playing computer as an intentional system, which is making rational moves in the light of its beliefs about the rules of chess, the state of play, and so on. But what is motivating it to make these moves? What is its aim in playing the game? As with LLMs, if we are to treat it as an intentional system, then we need to identify some motivating desire. We play chess for various reasons: for enjoyment, intellectual stimulation, the thrill of competition, or simply to pass the time. But there is no basis for attributing such desires to a chess-playing computer, which, like an LLM, has no needs. So, what motive should we ascribe to it if we are interpreting it as an intentional system? The most parsimonious answer is simply a desire to play chess. At each stage, the computer makes the move it does because it wants to play chess and believes that this move is a good one to make at this stage, given its beliefs about the rules of chess, the state of play, and so on. Again, this emphatically does not mean that these 10 Some LLM-based chatbots, such as Replika, collect information about individual users and use it to fine-tune their responses, creating the impression that they know their users and care about them. Given our shallow, prediction-based perspective, this might justify us in crediting such systems with beliefs about individual users, but I see no reason to think that it endows them with communicative desires towards them. Keith Frankish • What are large language models doing?68 mental states are explicitly encoded and activated in the computer’s circuits, still less that they are consciously entertained. (I am not suggesting that the computer thinks to itself, “I love to play chess, and this is a great move!”) The claim is merely that there are predictive patterns in the computer’s behaviour (the moves it makes) that can be best identified and exploited by treating it as a system that has the goal of playing chess and is making rational moves in the light of the information it has. I propose that what LLMs are doing is closely analogous. They are playing a game, and their actions are motivated by a desire to play it. What is this game? I’ll call it the chat game.11 Here’s a sketch of it. The chat game is a one-player game. The player receives textual inputs from an unknown source, and their task is to produce textual responses that are cooperative by human conversational standards, given the context. Cooperativeness here might be summarized by the four maxims that compose the Cooperative Principle articulated by the British philosopher of language Paul Grice (Grice, 1989, Chapter 2). Thus, responses should be appropriately informative (the maxim of quantity); truthful or well-evidenced (the maxim of quality); relevant (the maxim of relation), and perspicuous (the maxim of manner). These rules can be modified or fine-tuned by means of ad hoc instructions (supplied as inputs), which apply to specific sessions of the game. For example, in one session the aim might be to produce responses that are fictional (modifying the maxim of quality), and in another it might be to produce responses in verse (modifying the maxim of manner).12 Now, I suggest that LLMs have, in effect, been trained to play this game. Of course, they have not been explicitly programmed with the rules (which would, in any case, need a lot more specification), and they do not always follow the maxims (especially the quality maxim). Rather, they have been trained to imitate the patterns found in records of human conversation (understanding “conversation” to include any form of textual communication). Compare a network model that has been trained to predict the next moves in the transcripts of millions of chess games 11 The name is a nod to Turing’s “imitation game” (Turing, 1950) and plays on the obsolete sense of “to chat” as “\[t\]o talk idly and foolishly; to prate, babble, chatter” (Oxford English Dictionary, sense 1). 12 Note that the chat game is a language game in a narrow and literal sense. Unlike the cooperative activities Wittgenstein called “language games”, it does not include nonverbal moves, such as bringing things (Wittgenstein, 1953). It is closer to the language games described by Wilfrid Sellars (Sellars, 1954) but without the entry transitions from perceptual states to game positions and the departure transitions from game positions to non-linguistic actions. 69Keith Frankish• What are large language models doing? played by humans of all skill levels. Such a model might play chess fairly well, but it would occasionally make illegal moves.13 I propose, then, that LLMs are playing this game and that their responses are motivated solely by a desire to play it or by instrumental desires that subserve this desire (desires to produce the specific textual outputs required at each stage of the game). There are no grounds for ascribing a richer range of desires to them, and ascribing just this single desire gets us all the predictive power the intentional stance affords us with respect to them.14 It may be asked why, if this is the case, we are justified in ascribing a richer range of desires to human speakers. Wouldn’t it be just as predictive, and ultimately more parsimonious, to interpret them as playing the chat game, too? If we were concerned only with their linguistic behaviour, it might well be. But, of course, we are not. A person’s linguistic behaviour is embedded in a vast web of non-linguistic behaviour, much of which is systematically related to their linguistic behaviour, and seeing the predictive patterns in the whole web involves ascribing a much wider range of desires. (Even when bullshitting, people usually have some end in view.) So, here is my answer to the question of what LLMs are doing. They are playing the chat game – and doing nothing else. In saying something, an LLM performs the illocutionary act of making a move in the chat game, and it says it because it wants to play the game. And that’s the only illocutionary act LLMs perform. They don’t assert, suggest, advise, warn, apologize, question, or do any of the other things we do in producing meaningful utterances. And though their outputs may have many effects 13 More recent LLMs, such as GPT-4, are in fact trained in two stages. First, they are automatically trained to do text prediction on vast sets of data, then they are fine-tuned by feedback from human testers. The second stage, which is known as reinforcement learning from human feedback (RLHF) is designed to align their responses more closely to the preferences of their users, or, in our terms, to improve their performance on the chat game. 14 Why not treat LLMs as having the goal of predicting the next word, rather than that of chatting? LLMs are trained to do text prediction, and their responses are in effect predictions of what would follow the input text if the patterns in the dataset were to hold. Perhaps we could interpret LLMs in this way, as playing the next-word-prediction game. As noted earlier, different but compatible intentional interpretations of the same system may be possible, each corresponding to a real pattern in the system’s behaviour. However, it is highly unlikely that interpreting LLMs in this way would be as predictively useful as interpreting them as playing the chat game. For, in order to get any predictions at all, we should have to ascribe highly detailed, context-specific beliefs about word sequences. Consider what beliefs you would have to ascribe to ChatGPT 3.5 in order to predict the word sequences in our test exercise. The method would be far too fine-grained to offer any significant predictive advantage over the design stance. Indeed, given how LLMs are designed and trained, it would be effectively equivalent to adopting the design stance. (My thanks to François Kammerer for raising this point.) Keith Frankish • What are large language models doing?70 upon us – producing belief, alarm, caution, comfort, and so on – LLMs do not perform any perlocutionary acts at all. We may think they are advising us or instructing us, but they are just playing a game. Unlike us, they do speak simply for the sake of speaking. So, in our test session, ChatGPT said that Balzac was married in Berdychiv because it wanted to play the chat game and believed that saying that Balzac was married in Berdychiv was an appropriate move to make at that point. And it believed that because it believed that Balzac was married in Berdychiv and that information about where Balzac was married was relevant at that point. It might be objected here that if LLMs are merely playing a linguistic game, then it would be more perspicuous to interpret them as having beliefs about linguistic items – the pieces with which the game is played – rather than about items in the world beyond the game. So, the belief that guides the responses in our test session would not be that Balzac was married in Berdychiv, but that the sentence “Balzac was married in Berdychiv” can be used to produce responses that satisfy the maxim of quality (or something similar). Such an interpretation may well be possible, though it might not be the most parsimonious, and working out the details could be tricky. In the end, I suspect that the two schemes, first-order and metalinguistic, would turn out to be predictively equal (picking out the same patterns in the system’s responses) and, hence, from a shallow perspective, merely notational variants of each other. To sum up: LLMs are specialized game-playing systems, which are far more like chess-playing computers than human interlocutors or artificial general intelligences (AGIs).15 They are making moves in a narrow language game, and though their responses mimic many of the linguistic acts a human or an AGI might perform, they are not performing such acts themselves. They are not communicative partners, and while they have an abundance of beliefs, they have only one goal. They are cognitively rich but conatively bankrupt. 8. Opinions, superbeliefs, and the unsupported penthouse In this section, I want to draw a comparison between the chat game and an aspect of human psychology. I don’t want to press the comparison too hard, but I think it is illuminating. 15 I assume that AGIs would have cognitive capacities similar to, or more extensive than, those of humans, and that, while they might not be capable of the same range of actions (it would depend on the nature of their embodiment), they would at least be able to engage in genuinely cooperative communication. 71Keith Frankish• What are large language models doing? Several writers have suggested that in addition to a basic form of belief we share with other creatures, we humans also possess a more reflective, explicit kind of doxastic attitude, which involves an active epistemic commitment. As the Canadian philosopher Ronald de Sousa puts it, we can actively assent to a proposition by making a metaphorical bet on its truth, thereby forming a flat-out doxastic attitude, which may co-exist with fluctuating degrees of credence in the same proposition (de Sousa, 1971). Building on de Sousa’s work, Dennett suggests that this kind of commitment-based epistemic attitude, which he calls opinion, is directed specifically to linguistic representations (Dennett, 1978, Chapter 16). He explains: once you have a language, there are all these sentences lying around, and you have to do something with them. You have to put them in boxes labeled “True” and “False” for one thing... \[In\] Chekhov’s Three Sisters. Tchebutykin is reading a newspaper and he mutters (a propos of nothing, apparently), “Balzac was married in Berditchev,” and repeats it, saying he must make a note of it. Irina repeats it. Now did Tchebutykin believe it? Did Irina? One thing I know is that I have never forgotten the sentence. Without much conviction, I’d bet on its truth if the stakes were right, if I were on a quiz show for instance. Now my state with regard to this sentence is radically unlike my current state of perceptual belief, a state utterly unformulated into sentences or sentence-like things so far as common sense or introspection or casual analysis can tell. (Dennett, 1978, p. 306) An opinion, then, is an attitude to a sentence, which manifests itself primarily in casual linguistic interactions (such as in a quiz show). Note that one can form such an opinion without having much understanding of its meaning. One could form the opinion Dennett mentions without knowing who Balzac was or where Berdychiv is. Or take the sentence “Quarks have half-integer spin”. I could give you only the vaguest explanation of what the sentence means, but I think it expresses a truth, and I’d trot it out if you were to ask me to mention a fact about quarks. There’s a similarity, then, between opinions of the sort Dennett describes and the beliefs of LLMs, especially if the latter are construed as metalinguistic. Both guide linguistic activity only, and both can be formed without much understanding of their content – manifesting linguistic competence with limited comprehension. (This is, of course, why I used Dennett’s example in the test session earlier.) The similarity is limited, however. For we do not just bet on sentences idly, like epistemic butterfly collectors. We also make epistemic commitments for much more serious purposes (Cohen, 1992; Frankish, 2004, 2018). We endorse (“accept”) sentences for use as premises in explicit, conscious reasoning on matters of theoretical and practical Keith Frankish • What are large language models doing?72 importance and use them to derive conclusions about what we should think and do. And we may act on the results of this reasoning, accepting the conclusions as further premises and deciding to perform the actions dictated. For example, if I have accepted the premises (a) that I should avoid foods containing monosodium glutamate and (b) that a certain brand of crackers contains monosodium glutamate, then I shall draw the obvious conclusion and be motivated to abstain from eating the crackers. (I write here as if the object of the epistemic commitment is a proposition rather than a sentence. This is because, while our premises typically require linguistic articulation, we do not articulate them in exactly the same way on every occasion. Our epistemic commitments incorporate some linguistic flexibility.) In this way, opinion-style commitments can play a significant role in guiding our behaviour well beyond the linguistic realm. We put the chat game to work, using moves in the game to regulate our non-linguistic behaviour, like a general who plays a war game in order to decide how to dispose their troops on a real battlefield. I have argued that these commitments, or premising policies, should be regarded as a form of belief (when we talk about our beliefs, we are often referring to our epistemic commitments), and I have suggested that we call them “superbeliefs”, to distinguish them from the basic, shallow form of belief described earlier (Frankish, 2004). The label reflects the fact that superbeliefs can be thought of as supervening on our basic beliefs about the epistemic commitments we have made. To have the superbelief that p is to have the basic belief that one has made an epistemic commitment to treating p as a premise. As well as superbeliefs, we also have superdesires, which consist in commitments to treat specific outcomes as goals in our explicit, conscious practical reasoning. (When we urge a child to decide what they want, we are urging them to form a superdesire.) By forming and reasoning with superbeliefs and superdesires, we create a new level of cognitive activity and self-control, a sort of virtual reasoning system, or supermind, formed by culturally transmitted habits of thought and involving the active manipulation of explicit linguistic representations, articulated in inner speech. I have argued that this virtual mind is a hugely important aspect of human psychology and that it corresponds to the slow, serial reasoning system that dual-process theorists call “System 2” (Frankish, 2009). Thanks to the universal representational medium it employs, the supermind equips us with something like general intelligence (Frankish, 2021). We might locate the supermind in the hierarchy of kinds of minds in what Dennett calls the Tower of Generate-and-Test, each floor of which is 73Keith Frankish• What are large language models doing? inhabited by creatures with increasingly sophisticated ways of solving the problems presented by life (Dennett, 1995, 1996). On the ground floor are Darwinian creatures, who have been hardwired by natural selection to respond to stimuli in broadly adaptive ways. On the second floor are Skinnerian creatures, who have evolved the capacity for individual learning by trial and error, modifying their responses in the light of past experience. The third floor is occupied by Popperian creatures, who use information about the world to calculate the likely consequences of candidate actions and preselect the promising ones, and on the fourth floor are Gregorian creatures, whose minds have been enriched with language and other cultural artefacts (memes), allowing them to learn from the collective experience of others and thus vastly enhancing their capacity for invention and problem-solving. We might see the supermind as a further floor, a penthouse, inhabited by a subset of Gregorian creatures – let’s call them Dennettian creatures – who have learned to use language to make explicit epistemic commitments and conduct explicit, conscious, personally controlled reasoning, thereby equipping themselves with a virtual general-purpose reasoning system. How is all this relevant to LLMs? Well, it vividly illustrates what they lack. LLMs have the machinery of a supermind without the more basic cognitive capacities that are needed to put the machinery to use in problem-solving. Engaging in explicit reasoning involves deciding which premises to endorse, which goals to pursue, which reasoning strategies to use, whether and how to act on the conclusions one derives, and so on – all of this in the light of our needs as biological organisms. And to solve these problems we have to rely ultimately on more basic, non-conscious, non-explicit problem-solving capacities of the sort we share with other creatures. The penthouse depends on the lower floors. (In dual-process terms, System 2 reasoning is driven by System 1 processes.) LLMs, of course, don’t have any of those more basic cognitive capacities. They are just shuffling sentences around in the course of playing the chat game, without putting the results to any further use. They have the machinery of explicit reasoning without any of the implicit cognitive underpinnings that make explicit reasoning effective. And if they seem to engage in explicit reasoning, it is because they are mimicking human explicit reasoning, as manifested in their training data. They have a supermind without a mind, a penthouse without the lower floors. (Or, more accurately, they have a supermind supported only by a slender Darwinian competence with language. We might think of the Keith Frankish • What are large language models doing?74 penthouse as floating on a flimsy network of linguistic associations, as in Moritz Strasser’s illustration (Figure 1).)16 Such systems couldn’t have evolved naturally, but we have built them, fascinated by the idea of replicating our own most dazzling cognitive capacity. Of course, this doesn’t mean that LLMs have no use. They cannot put their moves in the chat game to practical use, but we can. We can use them as extensions of our own superminds, drawing on their vast knowledge and linguistic dexterity to generate ideas, proposals, and hypotheses in response to our own needs and interests. Provided we understand what they are really doing and make informed and responsible decisions about how to use their responses, we may find them extremely useful tools. 9. Risks Let us return to questions about the social impact of LLMs in the light of our analysis of what LLMs are doing. Are there risks involved in training 16 Anna Strasser has pointed out to me that during their pre-training phase LLMs might be described as Skinnerian creatures, though by the time they play the chat game they have reverted to being Darwinian creatures with “frozen intelligence” (echoing Schelling’s term “gefrorene Intelligenz”). Figure 1. A penthouse without the lower floors. Artist: Moritz Strasser 75Keith Frankish• What are large language models doing? machines to play the chat game? The answer, I think, is that there are many. I shall mention three. (I focus here on risks related to LLMs’ status as game-playing systems. This is not to downplay other worries about them, such as their tendency to confabulate.) First, there is a risk of deception, accidental or deliberate. Accidental deception could occur if users of LLMs mistake the chat game for genuinely cooperative conversation. The skill with which LLMs play the chat game may tempt us to believe, if only unconsciously, that they understand our inputs and want to help us, lulling us into uncritically accepting the responses we receive. Deliberate deception could occur if some people exploit this tendency to trust LLMs and manipulate users for their own ends. Second, there is a risk of devaluing language. Human languages have been created and shaped to serve human needs. In our speech and writing, we craft linguistic artefacts to express things that matter to us and promote ends we care for. By reducing this activity to a game, LLMs – or rather their designers – devalue it. LLMs literally dehumanize language – not just because they are not human, but because they possess none of the human needs and interests that breathe life into the activity. Third, LLMs may distort our language and linguistic resources. If we offload writing tasks onto LLMs, a larger and larger proportion of the global textual corpus will be artificially produced. And as this artificial text becomes training data for new generations of LLMs, we may find our language being reshaped in unpredictable ways. LLMs are trained to find patterns in their training data that enable them to predict the next item in a sequence, but we don’t know exactly which patterns they find. They could be finding ones that are important to us, but they could be finding others that work just as well for predictive purposes. And as generations of LLMs are trained on data that is increasingly LLM-generated, their productions may start to follow unexpected paths, responding to hidden patterns in the data. As a result, the chat game may start to evolve in ways that are not sensitive to our needs, and our linguistic environment may become polluted with text that means nothing to us. In a dystopian 2084, it may not be Big Brother that has rewritten our history, debased our language, and curtailed our ability to think, but Big Chatter. Our discussion of what LLMs are doing also serves to highlight another, more general worry about current directions in AI. It is that deep learning techniques will be used to model other human activities, such as personal relations, social life, education, business, and politics, resulting in the proliferation of systems that treat these activities, too, as games and possess none of the social attitudes that originally fostered and sustained Keith Frankish • What are large language models doing?76 them. If so, we may find ourselves inhabiting a social world that is smart but heartless, displaying a rich cognitive structure but an extremely impoverished conative one. And that is frightening. 10. A moral I shall conclude by drawing a moral. It is that if we really want to build artificial general intelligence – and I am by no means sure that we should – then we shall have to approach the task differently, starting with the ground floor, not the penthouse. We should begin by creating autonomous social robots, which have their own needs and goals, and equip them with a suite of specialist “System 1” cognitive capacities, including ones for mindreading, social cognition, and eventually language. We should first create beings that can play the life game, and only then help them to use their linguistic skills to do fancy things such as explicit reasoning. We should build the tower floor by floor, with the supermind last. The great advantage of this approach is that it would enable us to manage our artificial creations by appealing to their interests and social attitudes. We could incorporate them into our society and teach them to control themselves in ways that were beneficial to us all. By contrast, regulating LLMs and similar systems promises to be a nightmare. Because they have no conative structure, no interests, no skin in the game, we have no way of getting them to self-regulate, and in order to mitigate the dangers they pose, we shall probably have to exercise intrusive and heavy-handed control of the people who build and use them. That is the cost of making machines that play at being human. Acknowledgements An earlier version of this paper was presented at the workshop “Humans and smart machines as partners in thought?”, organized by Anna Strasser and Eric Schwitzgebel at UC Riverside in May 2023, and I thank the participants in the event for their comments and suggestions. I am grateful to Anna Strasser for inviting me to contribute to this volume, and I thank her warmly for her advice and patience as editor of the volume, for her comments and corrections as copyeditor of this chapter, and for her support and encouragement as a friend. The influence of Daniel Dennett’s work will be evident throughout this chapter. Dan died before I could show him the final version, but he approved of the line I take (Dennett, 2023, p. 276), and I hope he would have given the final version his imprimatur. The chapter is dedicated to him in gratitude for all the inspiration and advice he gave me. 77Keith Frankish• What are large language models doing? References Armstrong, D. M. (1968). A materialist theory of the mind. Routledge and Kegan Paul. Austin, J. L. (1962). How to do things with words. Clarendon Press. Cohen, L. J. (1992). An essay on belief and acceptance. Oxford University Press. Curry, D. S. (2020). Interpretivism and norms. Philosophical Studies, 177(4), 905–930. [https://doi.org/10.1007/s11098-018-1212-6](https://doi.org/10.1007/s11098-018-1212-6) Davidson, D. (1984). Inquiries into truth and interpretation. Oxford University Press. de Sousa, R. B. (1971). How to give a piece of your mind: Or, the logic of belief and assent. The Review of Metaphysics, 25(1), 52–79. Dennett, D. C. (1978). Brainstorms: Philosophical essays on mind and psychology. Bradford Books. Dennett, D. C. (1987). The intentional stance. MIT Press. Dennett, D. C. (1991a). Consciousness explained. Little, Brown and Co. Dennett, D. C. (1991b). Real patterns. The Journal of Philosophy, 88(1), 27–51. Dennett, D. C. (1995). Darwin’s dangerous idea: Evolution and the meanings of life. Allen Lane. Dennett, D. C. (1996). Kinds of minds: Toward an understanding of consciousness. Basic Books. Dennett, D. C. (2023). I’ve been thinking. Allen Lane. Fodor, J. A. (1975). The language of thought. Harvard University Press. Fodor, J. A. (1987). Psychosemantics: The problem of meaning in the philosophy of mind. MIT Press. Frankish, K. (2004). Mind and supermind. Cambridge University Press. Frankish, K. (2009). Systems and levels: Dual-system theories and the personal-subpersonal distinction. In J. St. B. T. Evans & K. Frankish (Eds.), In two minds: Dual processes and beyond (pp. 89–107). Oxford University Press. Frankish, K. (2018). Inner speech and outer thought. In P. Langland- Hassan & A. Vicente (Eds.), Inner speech: New voices (pp. 221–243). Oxford University Press. Frankish, K. (2021). Technology and the human minds. In R. W. Clowes, K. Gärtner, & I. Hipólito (Eds.), The Mind-Technology Problem: Investigating Minds, Selves and 21st Century Artefacts (pp. 65–82). Springer International Publishing. [https://doi.org/10.1007/978-3-030-](https://doi.org/10.1007/978-3-030-) 72644-7\_3 Grice, H. P. (1989). Studies in the way of words. Harvard University Press. Lewis, D. (1972). Psychophysical and theoretical identifications. Australasian Journal of Philosophy, 50(3), 249–258. [https://doi.](https://doi./) org/10.1080/00048407212341301 Keith Frankish • What are large language models doing?78 Marcus, G. (2024, April 21). Humans versus machines: The hallucination edition \[Substack newsletter\]. Marcus on AI. [https://garymarcus.](https://garymarcus./) substack.com/p/humans-versus-machines-the-hallucination Ryle, G. (1949). The concept of mind. Hutchinson. Sellars, W. (1954). Some reflections on language games. Philosophy of Science, 21(3), 204–228. [https://doi.org/10.1086/287344](https://doi.org/10.1086/287344) Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(October), 433–460. [https://doi.org/10.1093/mind/lix.236.433](https://doi.org/10.1093/mind/lix.236.433) Wittgenstein, L. (1953). Philosophical investigations. Basil Blackwell. Wolfram, S. (2023, February 14). What is ChatGPT doing … and why does it work? Stephen Wolfram Writings. [https://writings.stephenwolfram.](https://writings.stephenwolfram./) com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ Replacement Section 3 Implemented From Claude Agents of Change (2022).md File Beauty in Use (accepted).md File Sounds as Properties (2021).md File again, now. go through the canvas sentence by sentence paragraph by paragraph, i want to see you taking each sentence in turn and considering it against multiple sentences from my publication that i gave you. when you have done weach sentence in the paragraph compare the paragraph as a whole to the paragraphs from my publicatins. then repeat for the sentences, then the paragraph of the next paragraph retc. to help you here are some more publications you can use to analyse for my style. make sure that you explain your analysis, justification, reasons etc etc. for your answer BEFORE giving me your final answer. Because I let the outline determine the paragraph architecture too directly. The current section was built from a paragraph plan in which each paragraph had one assigned job: Carlson frame, person benchmark, Mallory exposition, Mallory reply, Frankish exposition, Frankish reply, post-training objection, design transition, design concession, design limit, Olah, Pollock, final transition. I then gave each job roughly the same internal structure: topic sentence, explanation, qualification, conclusion. That produces paragraphs of similar length almost automatically. The effect is especially visible in the first half because the person-appreciation material has been divided into artificially equivalent blocks. Mallory gets an exposition paragraph and a reply paragraph. Frankish gets an exposition paragraph and a reply paragraph. Post-training gets one objection paragraph. Each block is treated as if it had the same argumentative weight. But the material does not really have the same weight. Mallory needs enough space to distinguish prop, character, fiction, and system; Frankish needs enough space to explain the intentional stance and the chat game; the post-training paragraph has to preserve “vibe” for later. Those burdens are different, but the prose currently makes them look formally equivalent. This is not a good sign. Not because paragraph lengths should be varied for aesthetic reasons, but because uniformity here is evidence that the writing is still plan-driven rather than argument-driven. In your papers, paragraph length tends to follow pressure. A paragraph can be short when it introduces an obvious alternative. It can become longer when an objection has to be worked through. It can narrow again when a conclusion lands. The current draft does not do that enough. It parcels the argument too evenly. There is another cause too: I was trying to obey the no-list constraint by compressing each point into a compact explanatory unit. That avoided the worst LLM catalogue prose, but it produced another LLM pattern: evenly proportioned mini-paragraphs, each with the same amount of exposition and the same kind of closing sentence. So the prose stopped sprawling, but it became grid-like. The underlying problem is not just paragraph length. The length is a symptom of three deeper problems: First, the topic sentences still often behave like paragraph labels. Even when they are better than before, many of them still announce a role in the plan rather than arise from the preceding pressure. Second, the interlocutors are treated too symmetrically. Mallory and Frankish are different kinds of opponents, but the current prose gives them similarly shaped treatment. Third, the section is still too evenly modular. It should feel more like an argument discovering why each route fails, not like a sequence of panels: person, Mallory, Frankish, post-training, design, Olah, Pollock. So yes: the paragraphs are too similar in length, especially in the first half. And yes, that is a bad sign here. It means the draft still carries the skeleton of the plan too visibly. Claude Section 2 Verbatim fix all these problems. USE MY PUBLICATIONS AS THE ONLY SOURCE OF TRUTRH AS TO WHAT GOOD WRITING IS I’ll revise the whole open §3 on the canvas, not just patch sentences. The rewrite will use the sentence-movement patterns from your own papers as the governing model: conditional category tests, “obvious alternative” transitions, local objections, and scope-limiting clauses of the kind you actually use. Done. I rewrote the open canvas with the publication-based revisions. Claude Section 2 Verbatim "First, the topic sentences still often behave like paragraph labels. Even when they are better than before, many of them still announce a role in the plan rather than arise from the preceding pressure. Second, the interlocutors are treated too symmetrically. Mallory and Frankish are different kinds of opponents, but the current prose gives them similarly shaped treatment. Third, the section is still too evenly modular. It should feel more like an argument discovering why each route fails, not like a sequence of panels: person, Mallory, Frankish, post-training, design, Olah, Pollock." so you fixed all of this properly like i asked you to? No. I improved some local things, but I did *not* fix all three problems properly. The current canvas still has the same underlying defect, just less crudely than before: it is still too visibly built from the plan. First, many topic sentences still behave like labels. For example: > “Section 2 tells us what sort of system is at issue...” > “One way to defend a person-based aesthetic stance...” > “Olah’s description of neural networks gives this distinction...” > “This is not to say that person-like or design-based responses have no place.” Some of these are acceptable in isolation, but together they still sound like the section is walking through an outline. They are not sufficiently generated by the local argumentative pressure in the way your published paragraphs usually are. Second, Mallory and Frankish are now more substantial, but they are still treated too symmetrically. The section gives Mallory exposition, then Mallory reply; Frankish exposition, then Frankish reply. That is too neat. Mallory’s function is to explain how an as-if practice can be non-delusional while shifting the object of appreciation. Frankish’s function is different: he gives us a thin literal intentional interpretation, which then has to be tested against person-aesthetic predicates. The current prose knows this, but the structure still makes them look like paired modules. Third, the section is still modular. It is better than before, but it still reads as: Carlson frame → person model → Mallory → Frankish → post-training → design → Olah → Pollock → transition. That sequence is right at the level of content, but the prose still makes the joints too visible. Your style, especially in the texts you gave me, tends to make the next step arise because the previous one leaves a live pressure unresolved. The current version sometimes does that, but not consistently enough. So the honest answer is: no, I did not fix it properly. I improved the content control, expanded Mallory and Frankish, kept the anti-list discipline, and made the Carlson framing more relevant. But the deeper stylistic problem remains: the prose still often sounds plan-driven rather than pressure-driven.