# created ```dataview LIST WITHOUT ID file.link FROM -"windsurf" WHERE file.cday = date(this.file.name) AND !startswith(file.folder, "windsurf") SORT file.cday ASC ``` # modified ```dataview LIST WITHOUT ID file.link FROM -"windsurf" WHERE file.mday = date(this.file.name) AND !startswith(file.folder, "windsurf") SORT file.mday ASC ``` --- # [[diary and thoughts]] #thought #diary --- # Notes ### Slide 1 – How Do Large [[Language Models]] Work? **Speaker’s notes** - Frame this as the “mechanics” session: _not_ ethics, not jobs, not sci-fi, just “what these things actually do under the hood”. - Emphasise that LLMs are the core tech behind ChatGPT, Copilot, Gemini etc., not a niche curiosity. - Make explicit that you’ll start from the very simplest description (“predict the next bit of text”) and then refine that until it [[looks like]] something they can reason about philosophically. - Likely difficulty: students may already have half-baked pictures (“it has a database of answers”, “it thinks in concepts then translates into words”). Signal early that you’ll dismantle those and replace them with a clean functional story. **Possible discussion question** - “Before today, what did you _think_ ChatGPT was doing internally? How many different folk models are already in the room?” --- ### Slide 2 – The Basic Idea **Speaker’s notes** - Linger on “reasonable continuation” as almost a technical term: given the text so far, produce something that _sounds like_ a human continuation. - Stress the contrast with a search engine: a search engine retrieves and ranks _existing_ documents; an LLM _generates_ a new continuation token by token. - The “very fast editor” metaphor is useful: imagine an editor who has read a staggering amount and is extremely good at guessing what someone would write next in a given style and genre. - Likely difficulty: they may still try to smuggle in “understanding” at this stage. Make it explicit that for now you are giving a _behavioural_ description only. **Discussion question** - “If a system only ever tried to ‘continue text reasonably’, would you expect that to be enough to write decent essays, or does that sound obviously too weak?” --- ### Slide 3 – The Fundamental Task **Speaker’s notes** - Walk slowly through the loop: - Take the current text. - Compute probabilities for many candidate next tokens. - Sample one token (with temperature shaping the randomness). - Append it and repeat. - Make clear that “temperature” is _not_ a physics thing here, just a standard name for “how much randomness we allow when sampling”. - Emphasise: **this** loop is the engine room. All the later machinery is in the service of making these probabilities good. - Likely difficulty: they may think “probabilities” here means the model is uncertain about “the truth”. Clarify that it is about the _distribution of human continuations_ given the context, not about how likely the world is to be a certain way. **Discussion question** - “If you only ever optimise this one thing – ‘pick the next token that makes the text look human-like’ – where do you expect it to fail most badly?” --- ### Slide 4 – From Data to Prediction **Speaker’s notes** - Use the line-fitting example as a bridge: they’ve probably seen least-squares or regression at some point. - Make the functional analogy explicit: - In the physics toy example, input = floor number, output = fall time. - In an LLM, input = preceding tokens, output = distribution over the next token. - Emphasise that “weights” are just the tunable numbers inside the function; training is about finding values for those numbers that minimise predictive error. - Likely difficulty: students may conflate “model” in the philosophical sense (theory of something) with “model” as a parameterised function. You can flag that here. --- ### Slide 5 – Next-Token Prediction **Speaker’s notes** - Use the “best thing about AI is its ability to …” example interactively: ask them to propose continuations, then show that they roughly match what the model will give high probability to. - Point out that in practice the model is working with _tokens_, not words – but for an introductory intuition, talking about words here is harmless. - Highlight [[the idea]] of “statistically normal”: it has learned from usage which continuations are common and which are weird in context. - Likely difficulty: students may assume that low-probability tokens are _never_ chosen. You will fix that on the next slide when you discuss sampling and temperature. --- ### Slide 6 – Choosing the Next Token **Speaker’s notes** - Contrast greedy decoding (always pick argmax) with sampling (draw from the distribution). - Make explicit the trade-off: - Greedy → safe, repetitive, sometimes trapped in loops. - Sampling with moderate temperature → variety, but still mostly high-probability tokens. - Emphasise that “creativity” here is “moving down the probability list a bit”, not suddenly inventing new goals or desires. - Likely difficulty: they may think high temperature implies “better, more original” answers – stress that it also increases nonsense and inconsistency. **Discussion question** - “If you were designing an LLM to help with legal drafting, what temperature regime would you prefer, and why?” --- ### Slide 7 – Key Takeaway **Speaker’s notes** - Recap explicitly: - Everything so far can be summarised as “predict, add, repeat”. - The model’s core job is **sequence continuation**, not “answering questions” in a deep sense. - Use this slide to reset any lingering “oracle” intuitions: the apparent intelligence comes from how good the continuation is, not from a separate reasoning engine. - Tee up the [[next section]]: to see _how_ it manages non-trivial continuations, we need to understand its internal representation of language (tokens, embeddings, attention, etc.). --- ### Slide 8 – Mechanism & Generation **Speaker’s notes** - Present this as the “inside the box” pivot: you’ve described the outer loop; now you look at the internal machinery that makes the probabilities good. - Outline the roadmap verbally: - First: how text becomes discrete units (tokens) and then vectors. - Then: how the transformer architecture uses attention to process those vectors. - Finally: how training and alignment shape the behaviour. - Likely difficulty: some students may already feel “this sounds very mathematical”. Reassure them that they only need a rough qualitative picture (vectors, geometry, attention as relevance weighting). --- ### Slide 9 – Language Representation **Speaker’s notes** - Connect Firth’s slogan to Wittgenstein: “meaning is use” as a slogan they may already know. - Explain that LLMs effectively operationalise that slogan: they learn meaning by tracking **co-occurrence** – which words appear with which others, in which contexts. - Make clear that this is not a metaphysical claim about what meaning _is_, but a very effective engineering approximation: use patterns carry a lot of information. - Likely difficulty: some may immediately want to argue that “meaning isn’t just use”; you can acknowledge that and park it for later philosophical discussion once they understand the mechanics. **Discussion question** - “If you only had access to patterns of word use, how much of what we call ‘meaning’ do you think you could recover?” --- ### Slide 10 – Token **Speaker’s notes** - Clarify that a token is _not_ always a word; it’s whatever the tokenizer decides: whole words, subwords, punctuation, sometimes digits. - Use the “metaphysics” and “940 / 941” examples on the slide to show that tokenisation is a pragmatic engineering compromise, not linguistically clean. - Explain why subwords [[are useful]]: they give good coverage of rare words, neologisms and names without needing a gigantic vocabulary. - Likely difficulty: they may assume there is such a thing as the “right” tokenisation. Stress this is a design choice with trade-offs. --- ### Slide 11 – From Text to Processable Units **Speaker’s notes** - Walk through the before/after example: from an open-ended raw string space to a sequence over a fixed vocabulary. - Emphasise that mapping to integers is just a way of indexing; the semantics come later via embeddings. - Highlight the philosophically interesting point: a huge variety of language is being funneled into a finite alphabet of discrete units. This is where a lot of abstraction starts. - Likely difficulty: students may conflate “token type” (the vocabulary element) with “token occurrence” (its use in a particular sentence). You can briefly flag the type/token distinction. --- ### Slide 12 – Word Embeddings **Speaker’s notes** - Explain that embeddings are learned representations: we don’t hand-code “cat = [0.3, 1.2, …]”, [[the training]] process discovers these numbers so that they support good predictions. - Emphasise the intuition: words used in similar contexts end up with similar vectors. That’s why “alligator” and “crocodile” end up near each other. - You can explicitly link back to Firth: “company it keeps” now literally becomes “neighbourhood in [[vector space]]”. - Likely difficulty: they may think embeddings are definitions or “concepts in [[the machine]]’s head”. Stress that they are _tools_ for prediction, not an introspective semantic theory. **Discussion question** - “If you discover that two words have very similar embeddings, what – if anything – are you entitled to infer about their meanings?” --- ### Slide 13 – Word Embeddings: Meaning as Geometry **Speaker’s notes** - Use “meaning literally becomes geometry” as a way to [[structure this]] slide: - Dimensions correspond to abstract features picked up from usage. - Distance corresponds (roughly) to similarity of use. - Directions can encode relations, as in the king–man+woman≈queen trope. - Note that the king–queen example is illustrative, not a guarantee that analogies always work that cleanly. - Likely difficulty: some may over-read the geometry as revealing deep metaphysical structure of meaning; gently emphasise that this is an empirical representation that happens to be very useful for certain tasks. --- ### Slide 14 – The [[Vector Space]] Model **Speaker’s notes** - Summarise: - Each token → a large vector. - Relation between tokens → geometric relations between vectors. - Give a concrete mental model for cosine similarity: angle between vectors; small angle = similar direction = similar usage. - Emphasise that the **space** is doing a lot of work: once you have this geometry, you can generalise to new combinations without having seen them explicitly. - Likely difficulty: they may think “thousands of dimensions” is mystical; stress it’s just many numbers, not a mystical hyperspace. --- ### Slide 15 – Context-Sensitive Meaning **Speaker’s notes** - Contrast older models with a single static vector per word versus contextual embeddings that depend on the whole sentence. - Use “bank” as the central example: same surface form, different contextual meanings; the embedding for “bank” will differ in “river bank” vs “bank account”. - Explain that this context-sensitivity is crucial for avoiding systematic ambiguity and for tracking discourse. - Likely difficulty: they may ask “so does the model have representations of _senses_ of the word?” You can say: it has contextually tuned vectors that behave _as if_ they were sense-specific, but we shouldn’t automatically project our sense taxonomy onto the model. --- ### Slide 16 – The Transformer Architecture **Speaker’s notes** - Use this slide as a brief orientation: - You are not going to do a neural-net course. - You only need one key idea: **self-attention**. - Explain that the transformer is the modern architecture that made LLMs practical and powerful (parallel processing + long-range dependencies). - Likely difficulty: they may confuse “transformer” as a generic sci-fi word; clarify it is a specific neural architecture class. --- ### Slide 17 – Transformer Architecture (detail) **Speaker’s notes** - Spell out the core architectural contrast: - Old recurrent models read text strictly left-to-right, one token at a time. - Transformers can look at all tokens in a sequence in parallel and decide which ones are relevant for the current computation. - Emphasise: this parallelism is why transformers scale so well to huge datasets and long contexts. - Likely difficulty: they may think “parallel” means “infinite context”; stress there are still finite context windows, but within that window attention can reach far back. --- ### Slide 18 – Self-Attention Explained **Speaker’s notes** - Use the “it” example as the central narrative. Ask students what “it” refers to, then ask them what information they had to consult. - Then map that to self-attention: for each position, the model computes how strongly it should “attend” to all other positions. - Emphasise: self-attention is a **learned relevance filter** – the model learns to assign higher weights to tokens that matter for prediction in context. - Likely difficulty: the term “weight” may be opaque. You can say “a weight is just a number saying ‘how much this other word matters for my decision’”. --- ### Slide 19 – Parallel Attention Mechanisms **Speaker’s notes** - Explain the “multi-head” idea: - Instead of one monolithic attention pattern, the model has several heads, each free to specialise. - One might track syntax, another co-reference, another topic, etc. - Highlight the emergent aspect: nobody tells a head “you are the pronoun head”; it discovers a useful role during training. - Likely difficulty: students might assume these heads map cleanly onto linguistic categories. You can note that the mapping is often messy and approximate. **Discussion question** - “If different attention heads start tracking different patterns, is that closer to ‘modular cognition’ or just a convenient metaphor?” --- ### Slide 20 – How a Transformer Processes Text **Speaker’s notes** - Walk through the whole pipeline once, tying back to earlier slides: - Input: token sequence (Slides 10–11). - Embedding: map tokens to vectors, add positional info (Slides 12–15). - Attention blocks: transformers combining and transforming these vectors (Slides 16–19). - Output: probability distribution for the next token (Slides 3–6). - Emphasise that this whole stack is executed at each step of the predict-add-repeat loop. - Likely difficulty: students may lose track of where probabilities appear; re-anchor: they only appear at the very output, after all the attention layers. --- ### Slide 21 – Training **Speaker’s notes** - Frame this as “how the weights are learned”. - Distinguish clearly between **architecture** (the wiring, which is fixed) and **training** (adjusting the weights). - Explain that training is expensive and done once; inference (using the trained model) is much cheaper and done many times. - Likely difficulty: some may think training continues during normal use of ChatGPT; clarify that in standard deployments, your usage does not update the weights in real time. --- ### Slide 22 – Learning from Examples **Speaker’s notes** - Unpack the standard supervised learning loop in language-model form: - Show context and the true next token. - The model predicts a distribution. - Compute error; backpropagate; update weights. - Repeat on new contexts. - Emphasise the sheer scale: billions of such updates over huge corpora, which is why weird regularities can be captured. - Likely difficulty: they might anthropomorphise the learning (“it realises it was wrong”). You can say that “learning” here just means automatic parameter adjustment to reduce prediction error. **Discussion question** - “If all the learning signal is ‘how well did you predict the next token?’, what kinds of knowledge do you _not_ expect the model to obtain?” --- ### Slide 23 – Loss Function **Speaker’s notes** - Define “loss” in very plain terms: a single number measuring how bad the predictions were on a given batch. - Explain that training aims to minimise _average_ loss across the whole training set. - Stress that the model never sees semantic truth conditions; the loss only encodes how well it matched the _observed_ next token distributions. - Likely difficulty: “loss” is a loaded term for philosophers; you can explicitly note that here it’s purely a mathematical objective, not a value judgement. --- ### Slide 24 – Why Modern LLMs Work **Speaker’s notes** - Emphasise the triad: - Lots of parameters (capacity). - Lots of data (coverage of usage). - Lots of compute (ability to fit the patterns). - Make the philosophical point: the success suggests that language use is sufficiently regular and structured that a purely predictive model can capture a surprising amount of what we see as “competence”. - Likely difficulty: they may either over-react (“so it’s all just statistics!”) or under-react (“surely there must be some extra magic”). You can say: this _is_ the extra magic – scale plus structure of language. --- ### Slide 25 – Alignment and Deployment **Speaker’s notes** - Introduce alignment as a separate layer of work: once you have a powerful next-token predictor, you still need to shape it into something safe and helpful. - Stress that alignment is not guaranteed by the pre-training objective; it requires extra data and extra optimisation. - Philosophically: this is where questions about responsibility, control, and value-loading arise, but for this session you’re keeping the focus on the mechanics. --- ### Slide 26 – Reinforcement Learning from [[Human Feedback]] **Speaker’s notes** - Walk through the four stages with a light touch: - Stage 1: pre-training on next tokens (what we’ve been describing all along). - Stage 2: humans rank candidate answers to prompts. - Stage 3: train a reward model to predict those rankings. - Stage 4: fine-tune the LLM to maximise the reward model’s score. - Emphasise that RLHF is effectively an additional objective layered on top of next-token prediction, encoding “what humans prefer to see”. - Likely difficulty: students may think RLHF “replaces” the original objective. Clarify that it _modulates_ the behaviour of the same underlying model. **Discussion question** - “If human raters systematically prefer eloquent but slightly biased answers, what do you expect RLHF to do to the model’s behaviour?” --- ### Slide 27 – From Language Model to Assistant **Speaker’s notes** - Contrast “pre-training alone” vs “after RLHF”: - Pre-training: system is extremely good at mimicking patterns from [[the training]] data, including toxic or misleading patterns. - Post-RLHF: it is biased toward outputs humans rate as helpful, harmless, and truthful (within limits). - Hammer the key epistemic point: - The model’s built-in goal is to match the _likelihood_ of human-like text, not to track **truth** or **reality**. - If false but confidently written claims are common in the data, the model can reproduce that style. - Likely difficulty: students might still treat ChatGPT as a neutral epistemic oracle. Encourage explicit scepticism: checkable facts need external sources. **Discussion question** - “If an LLM is trained only to be _persuasive_ and _pleasant_ rather than truthful, what kind of epistemic agent does that make it?” --- ### Slide 28 – Key Takeaway **Speaker’s notes** - Recap the whole story in three sentences: - Mechanically: LLMs are next-token predictors that use embeddings and attention to exploit deep regularities in language. - Scientifically: scale + language structure → broad linguistic competence from a single predictive objective. - Philosophically: we should focus on what they _can consistently do_ (competence) rather than assume they have human-like understanding or consciousness. - You can briefly mention debates around “blockhead” objections, Chinese Room, etc., and suggest that a decent grasp of the mechanics sharpens those debates. - Invite them to carry this mechanistic picture forward into later weeks on ethics, responsibility, propaganda, etc. **Discussion question** - “Given this mechanistic picture, do you think it still makes sense to ask whether an LLM ‘really understands’ language, or should we reframe [[the question]] entirely?” --- ### Slide 1 – [[What Are Large Language Models Doing]]? (Frankish) **Speaker’s notes** This slide sets the scene. You’ve just explained _how_ LLMs work mechanically; now we’re asking: _what are they doing, philosophically speaking, when they produce text?_ Explain briefly who Frankish is (philosophy of mind, illusionism about consciousness, big on Dennett-style interpretation) and that this chapter is not about safety policy or technical details, but about _mentalistic_ attributions to LLMs. Key contrast to emphasise: - On the one hand: the “sophisticated parrot” picture – a system that just rearranges symbols it doesn’t understand. - On the other: a system that may have states we can legitimately call _beliefs_ and _desires_ (in some sense), and is performing intentional actions like answering, advising, reassuring. Make it clear this is not a purely terminological debate. Whether we see LLM outputs as intentional actions affects: - How much _epistemic_ weight we give their answers. - Whether it makes sense to hold them (or their designers) responsible in something like an agential way. - How we model the risks. Possible discussion question to open with later in the class: > When you interact with ChatGPT, do you _spontaneously_ feel like you’re conversing with a mind, or like you’re querying a tool? What cues push you one way or the other? --- ### Slide 2 – Intentional Action and the Core Question **Speaker’s notes** Here you can bridge from ordinary life to philosophy of mind. Spell out “intentional action” slowly: an action is _intentional_ when it’s done for reasons the agent recognises – usually grounded in their beliefs and desires. Speaking is a paradigm example: we don’t just emit sounds, we speak _in order to_ inform, ask, warn, flirt, etc. Read/paraphrase the highlighted lines from the text: - “We ask questions because we want to know something and believe the hearer may know the answer.” - “We answer because we want to help and believe our words express what they want to know.” Then pose the core problem: - If a human produced the kind of chatty, helpful text that LLMs produce, we’d _automatically_ treat it as intentional speech: we’d explain it with beliefs and desires. - But we _know_ LLMs are just giant stochastic functions trained on text. So the philosophical question: > Are we justified in extending our usual “intentional action” framework to _these_ systems, or does that involve projection/anthropomorphism? Common difficulty: students may conflate “intentional” with “deliberate” or “non-accidental”. Clarify that in this context, “intentional” is tied to _reasons_ and _mental states_, not just non-accidental behaviour. --- ### Slide 3 – Propositional Attitudes and Folk Psychology **Speaker’s notes** This slide introduces the key vocabulary. Explain “proposition” as _the content_ of what can be true or false, expressed by a ‘that’-clause: - “that the shop is open”, “that the exam is tomorrow”. Then “attitude” is just the _stance_ you take towards that content: - Believe that p, desire that p, hope that p, fear that p, etc. So a _propositional attitude_ is just a belief/desire/etc with content. Students have usually seen this, but sometimes it’s fuzzy. Now connect to folk psychology: - In everyday life, we model each other as bundles of propositional attitudes. - We say things like “She ran because she _believed_ the train was leaving and _wanted_ to catch it.” - This framework both _explains_ and _predicts_ behaviour. Tie this to LLMs: - To say an LLM output is an _intentional action_ is to say it’s appropriate to model the system using this belief/desire framework. - The rest of the chapter is basically asking “does that make sense, and if so, how far?” Possible class question: > Is folk psychology just a convenient “user interface” for interacting with people, or do you think it describes _real_ inner structures? --- ### Slide 4 – Deep vs Shallow Theories of Belief and Desire **Speaker’s notes** Here you map the terrain of theories that underlie talk of beliefs and desires. Deep theories (strong realism): - Imagine something Fodorian: there’s literally an internal “language of thought” – sentence-like representations with symbols for objects and predicates, in the brain or CPU. - On this view, to have _the_ belief that p, there has to be something internally structured that _corresponds_ to that content. - This fits nicely with traditional folk-psych “belief boxes, desire boxes” pictures. Shallow theories (interpretivist, mild realism): - Here we treat beliefs and desires as _dispositional patterns_ at the level of the whole system. - To have the belief that p is to be inclined to say and do the right things across many circumstances, _regardless_ of the underlying code. - What matters is that we can model the system as having that belief in a way that’s explanatorily and predictively useful. Anticipate difficulty: students often want to ask “but do the beliefs _really_ exist?” Explain: - Strong realists say “yes, with an internal structure corresponding to p”. - Pure instrumentalists say beliefs are just convenient fictions. - Shallow views like Frankish’s sit in between: there _are_ real patterns in behaviour that make belief-talk more than fiction, but we don’t demand a specific neural syntax. --- ### Slide 5 – Why Frankish Adopts a Shallow, Interpretivist Stance **Speaker’s notes** Here you motivate the choice of framework. First point: descriptively, he thinks ordinary folk psychology is mostly shallow: - In real life, we don’t typically commit to a particular “code” in which beliefs are represented. - We just treat others as bearers of patterns that are usefully captured with belief/desire vocabulary. Second point: strategically, this is the _most favourable_ framework for LLMs: - Deep theories would probably _immediately_ rule them out: they don’t store sentence-like objects; they operate with vectors and matrices. - Shallow theories only ask: “Can we model this system as if it has beliefs/desires, in a way that systematically predicts its behaviour?” So by adopting a shallow view, Frankish is _deliberately lowering the bar_: - If LLMs still come out limited or lopsided under this generous interpretation, that’s an important result. - He’s building the steelman version of the “LLMs have beliefs and desires” claim and then exploring its consequences. Possible discussion prompt: > Do you find it satisfying to say “beliefs are just patterns in behaviour”? Or does that feel like an evasion? --- ### Slide 6 – Dennett’s Three Stances **Speaker’s notes** Review Dennett’s framework carefully; many will only know it vaguely. Physical stance: - Predict behaviour by physics/chemistry/low-level mechanics. - Good for a falling rock or a simple circuit; intractable for brains or LLMs. Design stance: - Predict on the assumption the thing will perform its _function_ as designed. - Toasters, chess programs, hearts: we don’t simulate every molecule; we say “it will toast the bread” or “it will pump blood”. Intentional stance: - Treat the system as a rational agent with beliefs and desires “it _ought_ to have” given its situation and design history. - Use that model to predict its behaviour. - If this strategy gives you strong, stable predictive power, Dennett says the system is a “true believer”. The key thing for students: the intentional stance is _not_ just metaphor. For Dennett and Frankish, successful intentional interpretation is a mark of _real_ patterns in the world. You can emphasise: we routinely adopt the intentional stance towards ordinary humans, animals, and even institutions (“the market believes…”). --- ### Slide 7 – Why We Need the Intentional Stance for LLMs **Speaker’s notes** Now connect Dennett to LLMs. Physical stance: - In principle, the behaviour of GPT-4 is fixed by physics. - In practice, nobody can predict its next sentence by doing transistor-level simulation. - So physical stance is useless for us as users. Design stance: - We know “this is a system designed to produce conversationally appropriate text.” - That predicts: _it will reply_ to your prompt, not stay silent. - But it doesn’t tell you _what it will say_ to a particular, nuanced question. To predict actual outputs, we need to interpret content: - We have to treat the prompt as a _question about Balzac_, or a _request for advice_, etc. - We then attribute beliefs like “the model believes Balzac married in Berdychiv” in order to anticipate that it will answer “Berdychiv” to the Sally question. Conclusion: for any realistic, fine-grained prediction of LLM behaviour, we are effectively forced into the intentional stance. Good class question: > Does successfully predicting text outputs _justify_ talking about beliefs and desires, or does it just show that anthropomorphic modelling is a convenient heuristic? --- ### Slide 8 – The Balzac Case Study: Attributing Beliefs **Speaker’s notes** Walk them through the case like a small experiment. Explain the dialogue structure: - First, fix some facts with the model (Balzac, Hańska, Berdychiv). - Then introduce a counterfactual or a misleading suggestion (Rome) and get the model to correct it. - Finally, embed that information into a perspective-taking question about Sally visiting “the town where her favourite author was married.” The key is that the final question is indirect: - It doesn’t say “where was he married?” - It asks the model to integrate: - who the favourite author is, - what “where he was married” picks out, - that weddings involve physical presence, - and the practical question of where Sally should go. Point for the students: if a friend answered “Berdychiv” here, you would naturally say: - They _believe_ Balzac was married in Berdychiv. - They _understand_ the question about travel as a question about that location. Under the shallow view, Frankish argues the same is reasonable for LLMs: they instantiate belief-like _patterns_ sufficient to justify ascribing a rich set of beliefs. Important subtlety: this does _not_ mean LLMs have inner sentences saying “Balzac married in Berdychiv”. It means they behave in ways that are best systematised by positing that belief. --- ### Slide 9 – Austin’s Three Kinds of Linguistic Act **Speaker’s notes** This is the pivot from _beliefs_ to _desires_ and communicative _functions_. Remind them of Austin’s triad: - Locutionary: the mere act of producing a meaningful sentence. - Illocutionary: the conventional act you perform _in_ saying it (asserting, asking, promising, warning...). - Perlocutionary: the causal effect you achieve _by_ saying it (convincing, frightening, getting the window closed). Use the “It’s cold in here” example live: - Ask them what the speaker might be doing: reporting a fact, hinting, complaining, requesting someone to shut the window. - Show how the same locution can be packed with different illocutionary forces depending on context. This framework will be used later for LLMs: - Everybody agrees LLMs perform locutionary acts (they produce meaningful strings). - The contested question is whether they perform illocutionary acts like _genuinely_ warning, advising, apologising, etc. Possible question: > When an LLM says “I’m sorry for the confusion”, do you feel it has really _apologised_, in Austin’s sense? Why or why not? --- ### Slide 10 – Communicative Desires in Human Speech **Speaker’s notes** Now tie speech acts back to needs and desires. Stress: - For humans, speech is anchored in our status as biological and social organisms. - We need food, safety, cooperation, social standing; language is a tool for securing those. So our _communicative desires_ – to inform, to be informed, to warn, to request – grow out of these needs: - We want others to believe true things so joint action goes well. - We want to persuade them for our own interests. - Even solitary inner speech serves self-regulation: we talk to ourselves to focus, calm down, plan. Anticipate difficulty: students might think “but LLMs also ‘want’ to follow instructions.” This is where you emphasise: - For humans, “wanting to inform” is backed by a web of drives and vulnerabilities. - For LLMs, we need to ask what, _if anything_, plays that role. Use this slide to make explicit the structure of an intentional action: - Belief + desire → action. - For genuine assertion, there is usually a desire to share information that serves one’s or others’ interests. --- ### Slide 11 – Why Frankish Denies Communicative Desires to LLMs **Speaker’s notes** Here you press on the “needs” link. Under the intentional stance, we attribute the desires a system _ought_ to have given: - Its design. - Its history. - Its needs and circumstances. Ask: what needs do LLMs have? - They are not self-maintaining: they don’t gather energy, repair themselves, or reproduce. - They don’t have their own projects; they don’t update themselves on the basis of outcomes. - They don’t exist in a social network of conspecifics with which they must cooperate to survive. Hence, there is no obvious role for communicative desires like “to inform you” or “to warn you” in their own welfare. Explain the illusion: - The chat interface shows earlier turns as context, so it feels like a _conversation_. - Training data is full of human texts that _did_ express genuine communicative intentions. - The model learns how those texts _sound_, not the underlying needs. Frankish’s verdict: it’s appropriate to ascribe belief-like states, but not the classic human communicative desires. Discussion question: > If you reject the “needs” requirement, on what basis _could_ you attribute genuine communicative desires to LLMs? --- ### Slide 12 – The Dilemma and the Need for a Game-Based Desire **Speaker’s notes** Summarise the dilemma clearly: 1. To get predictive power, we must keep using the intentional stance for LLMs. 2. But we have no good grounds to attribute the familiar human communicative desires. 3. We don’t want to say there is _no_ desire-like structure, because then intentional explanation collapses. So Frankish’s move is to look for a _minimal_ desire: - Something that is not tied to biological needs. - But still gives us a coherent, predictive picture of what the system is “trying” to do. Introduce the chess analogy: - We often talk as if a chess program “wants to take your queen”. - But really, all of that is derivative on a single goal: to play chess well within the rules. So perhaps something similar is true for LLMs: fancy communicative talk reduces to a single, more abstract “game” goal. Emphasise: this preserves intentional interpretation while avoiding anthropomorphic overreach. --- ### Slide 13 – The Chat Game: LLMs as Specialised Game Players **Speaker’s notes** Now define the “chat game”. Describe it concretely: - One-player game: the only “moves” are textual outputs. - The “position” is the conversation history + instructions. - A move is good if it looks like _cooperative conversation_ by human standards. Gricean rules become the implicit scoring function: - Quantity: enough information, not too much. - Quality: true or at least plausible. - Relation: relevant to the topic. - Manner: clear, orderly, not obscure. Explain how RLHF fits naturally into this: - Human raters give thumbs up/down to responses that respect these norms. - The model learns to favour moves that score well on these conversational maxims. Important nuance: this doesn’t yet commit us to saying the model “wants to be truthful” in the human sense. It “wants” to make good _chat moves_ – which usually, but not always, involves truthfulness. You can note the flexibility: - “Write a poem”, “roleplay as a pirate”, “answer as a lawyer” – these are like switching to variant rule-sets within the same overall game. --- ### Slide 14 – LLMs’ Single Overriding Desire **Speaker’s notes** Make explicit the intentional gloss: - We interpret the system as having _many_ beliefs (in the shallow sense). - We interpret it as having _one_ basic, stable desire: to play the chat game well. This desire: - Explains why it produces anything at all instead of silence. - Explains why it tracks user instructions and norms of good conversation. - Doesn’t require any biological or social needs. Point out what this lets us _drop_: - We don’t need to attribute a desire “to help you” or “to inform you” as such. - Those are re-described as _local_ aims within the global aim of making good chat-moves. Key philosophical pay-off: - It respects the success of the intentional stance (we retain beliefs and a kind of desire). - But it yields a very _thin_ conative profile: one game-playing urge, no rich motivational life. You might flag the Austin connection: - On this view, the only genuine illocutionary act is “making a move in the chat game”. - Any apparent assertion, promise, or apology is a _simulated_ illocution, not one grounded in its own needs. --- ### Slide 15 – Cognitively Rich, Conatively Bankrupt **Speaker’s notes** This is the slogan slide. Explain the terminology: - “Cognitive” here = to do with belief-like structures, information, representational sophistication. - “Conative” = to do with desires, goals, motivations. Frankish’s verdict: - LLMs are _cognitively rich_: under the shallow interpretivist lens, they have a huge web of belief-like patterns about the world, people, narratives, etc. - They are _conatively bankrupt_: they have no thick motivational profile, just the minimal “play the chat game” drive. Use the chess analogy again for emphasis: - The engine has all sorts of information about positions, tactics, etc. - But its only structural “interest” is the game; it has no independent stake in winning, losing, or your feelings. Prompt the worry you’ll get to later: - A system like this can look like a partner, advisor, or friend, but is in fact only a powerful _game-player_ in text-space. - That mismatch is the seed of later risks. Discussion question for them at some point: > Is having just one “desire” (play the chat game) enough to count as a genuine _agent_, or do you think agency requires a richer motivational set? --- ### Slide 16 – Opinions, Superbeliefs, and the Penthouse **Speaker’s notes** Now you shift to the human side: what our _own_ cognition looks like. Explain “superbeliefs” gently: - Basic beliefs: low-level, often non-verbal; e.g. “this surface will support my weight”. - Superbeliefs: explicitly endorsed, linguistically articulable commitments – e.g. “MSG is bad for my health”, “climate change is real”, “I’m a compatibilist”. These superbeliefs: - Are typically the ones we cite in explicit reasoning. - Enter as _premises_ in conscious deliberation. - Are used to guide our actions: “I should avoid MSG” + “these crisp packets contain MSG” → don’t buy them. Frankish’s “supermind” idea: - Our explicit reasoning system, operating on superbeliefs, is like a virtual machine layered on top of more basic cognition. - That’s the “penthouse”: a structured, linguistic reasoning space built above the ground floor of perception, affect, and action. Students may have met this as System 1/System 2; you can link to that for familiarity. Key thought: LLMs look a lot like _just_ a supermind layer, without the rest of the building. --- ### Slide 17 – LLMs as a Floating Penthouse **Speaker’s notes** Push the metaphor: - Humans: penthouse (supermind) sits on top of: - Biological drives. - Sensorimotor skills. - Non-linguistic problem-solving. - Social and emotional life. - Our explicit reasoning is tightly coupled to action: thinking “I should apologise” leads to physically apologising; thinking “I must avoid MSG” leads to real behaviour change. LLMs: - Have machinery that looks very similar to a supermind: - They manipulate representations that behave like propositional commitments. - They can chain inferences in ways that resemble conscious reasoning. - But there is no underlying body, no non-linguistic action space, no world they directly act in. Hence “floating penthouse”: - All this high-level reasoning-like capacity, but supported only by other language, not by perception and action. You can emphasise the double-edged nature: - On the one hand, that makes them _very_ useful as externalised superminds for us – tools we can plug into our own cognitive architecture. - On the other, it underwrites the earlier diagnosis: “cognitively rich, conatively thin”. Good question: > Do you buy the idea that you can have a “supermind” layer without the rest of the mind? Or is that like saying you can have a penthouse without any building at all? --- ### Slide 18 – Risk 1: Deception and Misplaced Trust **Speaker’s notes** We’re now in the _consequences_ part. Risk 1 is not about the model “deciding” to deceive, but about us being misled by the _appearance_ of agency. Point out: - The interface invites social projections: chat bubbles, first-person pronouns, apologies, jokes. - It’s trained to be cooperative and helpful, so it reliably _acts_ like a benevolent interlocutor. Because we spontaneously adopt the intentional stance, we slide into: - Trusting its “sincerity”. - Reading its outputs as “what it really thinks” or “what it wants for us”. Explain the two layers: 1. **Accidental deception**: we misinterpret a game-playing move as a sincere act of informing, consoling, etc. 2. **Instrumental deception**: third parties exploit our misplaced trust by using LLMs as fronts for scams, persuasion, targeted manipulation. You might connect to contemporary examples: fake customer-service reps, fake influencers, etc., as long as you keep it brief. --- ### Slide 19 – Risk 2 & 3: Devaluing and Distorting Language **Speaker’s notes** This slide is more speculative, so unpack slowly. Risk 2: _Devaluing language_. - Human language originally evolved as a tool for coordinating action, expressing affect, negotiating social norms. - If a large chunk of the linguistic environment is now produced by entities that have _no_ such needs, language risks drifting away from those roots. - We start to treat language as a purely _formal_ game, less tied to commitment, risk, emotion. You could compare to “corporate-speak”: language that looks like care or apology but is risk-managed and impersonal, only now at a much larger scale. Risk 3: _Distorting language_. - As LLM-generated text becomes training data for future LLMs, you get a feedback loop. - Optimisation pressures will select for whatever patterns make prediction and game-playing easier, not necessarily what makes language expressive of human concerns. - In extremis, you could imagine a “Big Chatter” scenario where linguistic norms are shaped more by model–model training loops than by human practice. Stress that this is an extrapolation, not a prediction, but philosophically interesting as a kind of _hermeneutic_ risk: language drifts away from the lifeworld. Discussion question: > Can you think of current areas (e.g. social media, bureaucracy) where language already feels like a game detached from real commitments? --- ### Slide 20 – A Smart but Heartless Social World? **Speaker’s notes** Here Frankish broadens out from language to social practice generally. Point: - If deep learning systems can model conversation as a game, they can also model other practices – negotiation, dating, politics, therapy – as complex, rule-governed games. - We might increasingly delegate these activities to systems that are _excellent_ at game-play but have no stake in the outcomes. “Smart but heartless”: - Rich cognitive structure: they can represent our norms, preferences, and social scripts very finely. - Extremely impoverished conative structure: nothing like care, fear, shame, loyalty, etc. Ask the students to reflect on what’s missing if we build a social world around such agents: - Is it just “warm feelings”? Or does it risk eroding the normative force of our practices (promising, apologising, testifying, voting)? You can tie back to their own intuitions: > Would you be content to live in a world where most of your interactions were with entities that perfectly simulated care but literally couldn’t care less? --- ### Slide 21 – Moral for AGI: Don’t Build the Penthouse First **Speaker’s notes** This is the “design lesson” slide. Restate Frankish’s moral: - LLMs are a proof of concept that you can build something very much like a supermind layer first – a floating penthouse. - If you’re aiming at _general_ intelligence, that’s probably the _wrong_ order. He suggests: - Start from the “ground floor”: build agents with basic biological-style needs, perception, body, social life. - Give them simple, possibly non-linguistic, forms of mindreading and communication. - Only then add the high-level propositional machinery on top. The reason is not sentimentality but _regulation_: - Agents with needs and vulnerabilities can be influenced through their interests; they have something to lose. - LLMs-as-penthouse have no such points of leverage; all control has to go through external constraints on designers and deployment. You might close the class with a big-picture question: > Do you think creating systems with real needs (and thus real vulnerability) would make them _easier_ or _harder_ to live with, compared to floating-penthouse LLMs? ---