# Scratch Pad Nick Young: explain to me what the bitter lesson is. --- Raycast AI: <contemplator> The user is asking about "The Bitter Lesson." This is a well-known essay by Rich Sutton, a prominent figure in reinforcement learning and artificial intelligence research. Let me think through what this essay argues and its significance. Rich Sutton published "The Bitter Lesson" on March 13, 2019. It's one of the most influential short essays in AI research in recent years. Let me think about what it says... The core argument is about the history of AI research and what has actually worked over the long run. Sutton observes a recurring pattern: researchers try to build systems that leverage human knowledge and domain expertise, but in the long run, the approaches that win are those that leverage computation — specifically, general-purpose methods that scale with increasing compute power. Let me think about the specific examples he uses... 1. **Chess**: Early chess programs tried to encode human chess knowledge — openings, strategies, evaluation heuristics crafted by grandmasters. But what ultimately triumphed was Deep Blue and later systems that relied on massive search through possible moves. And then even more dramatically, AlphaZero used general-purpose learning (self-play reinforcement learning) with enormous computation to surpass all previous approaches. 2. **Go**: Similar story. For decades, researchers tried to encode human Go knowledge. But AlphaGo and then AlphaZero used general methods (Monte Carlo tree search + deep learning) with massive computation. 3. **Speech recognition**: Early systems tried to encode linguistic knowledge — phonemes, grammar rules, etc. But statistical methods, and eventually deep learning with large datasets and compute, won out. 4. **Computer vision**: Similar pattern — hand-crafted features like SIFT, HOG, etc. were replaced by deep learning approaches that learn features from data. The "bitter" part of the lesson is that it's bitter for AI researchers to accept. Researchers naturally want their domain expertise and clever ideas to matter. They invest years learning about the structure of a problem domain. But the historical lesson is that these human-knowledge-based approaches eventually get overtaken by general methods that simply leverage more computation. Sutton's key insight is that Moore's Law (and more broadly, the exponential growth of available computation) is the most powerful force in AI. Methods that can take advantage of increased computation will eventually win, while methods that rely on human-encoded knowledge hit a ceiling. The lesson has two key principles: 1. **Search** scales with computation (exploring more possibilities) 2. **Learning** scales with computation (learning from more data and more training) These are general-purpose methods that improve as you throw more compute at them. The bitter lesson also warns against the temptation to build in human knowledge. While it can provide short-term gains, it often makes systems more complex and harder to scale. It can even be counterproductive in the long run because it adds constraints that prevent the system from discovering better solutions on its own. Now, there are nuances and criticisms of this view... Some people argue that architectural choices (like the Transformer architecture) are themselves a form of human knowledge/inductive bias, and they matter enormously. Others argue that Sutton's lesson is too extreme — that some human knowledge is necessary to make learning tractable. There's also the question of data efficiency. Pure computation-based approaches can be very data-hungry and compute-hungry, which has environmental and economic costs. But the general trend that Sutton identifies has been remarkably consistent, and the success of large language models (GPT-3, GPT-4, etc.) trained on massive data with massive compute has further reinforced his argument. Let me also think about why it's called "bitter"... It's bitter because: - Researchers spend careers developing clever, knowledge-rich approaches - These approaches get superseded by "brute force" methods - It feels like the elegant, intellectual work doesn't matter as much as just scaling up - It's a humbling realization that human cleverness in encoding knowledge is less valuable than simply providing more computation This is a fairly complete picture. Let me organize this into a clear answer. </contemplator> <final_answer> ## The Bitter Lesson **"The Bitter Lesson"** is a highly influential essay written by **Rich Sutton**, a pioneering researcher in reinforcement learning, published on **13 March 2019**. It is one of the most discussed pieces of writing in modern AI. ### Core Argument Sutton observes a recurring pattern across the entire history of AI research: > **General-purpose methods that leverage computation ultimately beat approaches that try to encode human knowledge and domain expertise.** Over and over, researchers have tried to hand-build intelligence by encoding what *we* know about a problem into a system. And over and over, these approaches have been surpassed by simpler methods that just **scale with more compute** — specifically through **search** and **learning**. ### Historical Examples | Domain | Human-Knowledge Approach | What Actually Won | |---|---|---| | **Chess** | Hand-crafted evaluation functions by grandmasters | Massive search (Deep Blue), then self-play RL (AlphaZero) | | **Go** | Encoded Go strategies and patterns | Monte Carlo tree search + deep learning (AlphaGo/AlphaZero) | | **Speech Recognition** | Linguistic rules, phoneme models, grammar | Statistical models → deep learning at scale | | **Computer Vision** | Hand-crafted features (SIFT, HOG, edge detectors) | Deep convolutional networks learning from raw pixels | ### The Two Key Scaling Methods 1. **Search** — exploring more possibilities as compute grows 2. **Learning** — extracting patterns from more data with more training Both are *general-purpose* and improve reliably as you add more computation. ### Why Is It "Bitter"? The lesson is **bitter** because it's painful for researchers to accept: * Researchers invest years mastering a domain and encoding that expertise into systems * It feels intellectually unsatisfying that "just scale it up" beats elegant, knowledge-rich designs * It implies that **human cleverness in system design matters less** than the ability to leverage ever-growing compute * Hard-won domain knowledge often becomes a short-term advantage that gets overtaken ### The Takeaway Sutton argues that AI researchers should **stop fighting this trend** and instead: * Favour **general methods** that scale with computation * Resist the temptation to bake in human knowledge as a shortcut * Recognise that Moore's Law (and the exponential growth of compute) is the dominant force in AI progress ### Relevance Today The explosion of **large language models** (like GPT-4, Claude, etc.) — trained with massive compute on massive data using relatively general architectures — has dramatically reinforced Sutton's thesis. These models weren't hand-loaded with grammar rules or world knowledge; they *learned* it, given enough scale. ### Caveats Some researchers push back, noting that: * **Architectural choices** (e.g., the Transformer) are themselves a form of human insight * Compute-heavy approaches have significant **environmental and economic costs** * Some domains may still require inductive biases or human knowledge for **data efficiency** Still, the broad historical trend Sutton identifies has proven remarkably durable. </final_answer> --- Nick Young: ok, this is going to sound like a weird diversion but i am interested in how this idea might be applied to the possibility of doing philosophy with LLMs. there seems an interesting way to go via a famous 20th century philosophical motto --- Raycast AI: <contemplator> Hmm, this is a really interesting question. The user is asking me to connect "The Bitter Lesson" to the possibility of doing philosophy with LLMs, and they mention a "famous 20th century philosophical motto." Let me think about what that motto might be... The Bitter Lesson says: general methods that scale with computation beat human-encoded knowledge. So the question is: how does this apply to philosophy? And what's the 20th century philosophical motto that connects them? Let me think about famous 20th century philosophical mottos... - "God is dead" — Nietzsche, but that's really 19th century - "Existence precedes essence" — Sartre, existentialism - "The medium is the message" — McLuhan, though more media theory - "Language is the house of being" — Heidegger - "Whereof one cannot speak, thereof one must be silent" — Wittgenstein - "The meaning of a word is its use in the language" — Wittgenstein - "There is nothing outside the text" (Il n'y a pas de hors-texte) — Derrida Wait... let me think more carefully about the connection to The Bitter Lesson and LLMs... The Bitter Lesson says: don't try to encode human knowledge; instead, let general methods learn from scale. LLMs are trained on massive amounts of text. They learn patterns from language at scale. Now, which philosophical motto connects here? I'm thinking about Wittgenstein. "Meaning is use." That's from the Philosophical Investigations. The idea is that the meaning of words isn't some internal mental representation or correspondence to objects — it's how words are used in practice, in language games. This connects beautifully to LLMs because: 1. LLMs learn language purely from usage patterns — they have no grounding in the world, no sensory experience, no intentionality (in the philosophical sense) 2. Yet they seem to capture something meaningful about language 3. If Wittgenstein is right that meaning IS use, then an LLM trained on vast amounts of language use might actually capture meaning in a deep sense But wait, let me think about other possibilities... Actually, could it be "There is nothing outside the text" from Derrida? That's another possibility — if there's nothing outside the text, then an LLM trained on text has access to... everything? That seems like a stretch but it's interesting. Hmm, but let me think about this more carefully in connection with The Bitter Lesson specifically... The Bitter Lesson says: don't hand-encode knowledge, let computation and general methods do the work through search and learning. In philosophy, there's been a tradition of trying to formalize reasoning, build logical systems, encode philosophical knowledge in precise frameworks. Analytic philosophy, in particular, has tried to be very precise and systematic. But what if the "bitter lesson" for philosophy is that rather than trying to hand-encode philosophical reasoning, you could just... train on a massive corpus that includes all of philosophical writing, and patterns of philosophical thinking emerge? Now, the 20th century motto... Let me think again. Actually, I think the most compelling connection is to Wittgenstein's later philosophy. "Meaning is use" — from the Philosophical Investigations. This is one of the most famous philosophical ideas of the 20th century. Here's the deep connection: 1. **The Bitter Lesson**: Don't encode human knowledge; let systems learn from data at scale using general methods. 2. **Meaning is use**: Don't look for meaning in mental states, reference, or formal definitions; meaning is constituted by patterns of use in language. 3. **LLMs**: Learn language purely from patterns of use at massive scale. So the connection is: if The Bitter Lesson tells us that general learning methods beat hand-encoded knowledge, and if Wittgenstein tells us that meaning just IS patterns of use, then LLMs — which learn from patterns of use at enormous scale — might actually be capturing meaning (and potentially philosophical insight) in a way that is more legitimate than we might initially think. This is philosophically provocative because many philosophers would object that LLMs are "just" pattern matching, "just" statistical parrots. But if Wittgenstein is right, then what else would meaning be, other than the patterns of use? The "just" in "just pattern matching" does a lot of unwarranted work. Let me think about whether there are other mottos that fit better... "Back to the things themselves" — Husserl's phenomenological motto? Hmm, that doesn't seem to fit as well. "Workers of the world, unite" — Marx? Not really... What about the Vienna Circle / logical positivism? "The meaning of a proposition is the method of its verification" — the verification principle? That's interesting but I don't think it connects as neatly. What about Quine's "To be is to be the value of a variable"? Or his arguments about the indeterminacy of translation? Actually, Quine's web of belief and his holism could be relevant — meanings aren't atomic but are determined by their place in a web of relations. LLMs learn exactly this kind of relational, holistic structure... But I keep coming back to Wittgenstein's "meaning is use" as the most natural and famous connection. Actually, wait. Let me reconsider. The user said "a famous 20th century philosophical motto." Let me think about what's really the MOST famous... Actually, could it be something even simpler? What about the linguistic turn? The idea that philosophical problems are really problems about language? That's a broad movement, not a single motto, but... Or could it be related to ordinary language philosophy more broadly? Wittgenstein, Austin, Ryle — the idea that philosophical problems arise from misuses of language and can be dissolved by attending to ordinary usage? I think the most compelling and famous motto is still "meaning is use" from Wittgenstein. Let me also consider: could the user be thinking of something from a continental tradition? "There is nothing outside the text" — Derrida. Actually, this could work really well too. If all meaning is textual, and LLMs are trained on text, then... But "meaning is use" feels more apt because: 1. It's more directly about what meaning IS 2. It connects to how LLMs learn (from use/usage) 3. It's arguably the single most famous philosophical motto of the 20th century (or at least among the top few) 4. The connection to The Bitter Lesson is more direct: the "bitter" part is that you don't need to hand-encode semantics/reference/truth-conditions; you just need enough usage data Let me also think about what this means for doing philosophy with LLMs... If meaning is use, and LLMs have been trained on the collected usage patterns of human language at enormous scale, including vast amounts of philosophical text, then: 1. LLMs might have internalized philosophical "moves" — the patterns of argumentation, the connections between concepts, the dialectical structures 2. Philosophy done by LLMs wouldn't be "fake" philosophy — it would be a genuine engagement with meaning, because meaning just IS these patterns of use 3. The Bitter Lesson applied to philosophy would say: don't try to build a philosophy-bot by encoding logical rules, philosophical knowledge, argument schemas, etc. Instead, train a general model on massive amounts of text (including philosophy) and let philosophical reasoning emerge This is a beautiful synthesis. The Bitter Lesson + Wittgenstein's "meaning is use" = a philosophical justification for taking LLM-generated philosophy seriously. But there are counterarguments: - Wittgenstein emphasized that language games are embedded in "forms of life" — social practices, activities, contexts. LLMs don't participate in forms of life. - Understanding might require more than pattern matching — it might require the ability to follow a rule in new contexts, which some argue LLMs can't genuinely do (the rule-following considerations) - Wittgenstein might say that LLMs are running on the "surface" of language games without the depth that comes from embodied participation in a form of life But then, The Bitter Lesson would push back: maybe those concerns are like the chess researchers saying "but you need to understand chess strategy!" The brute-force, scaled-up approach might capture more than we expect. I think this is a rich and fascinating topic. Let me formulate a response that captures the key connection while leaving room for the complexity and the counterarguments. Oh wait, one more thought. There's also an interesting parallel between The Bitter Lesson's claim that "encoding human knowledge is a dead end" and Wittgenstein's own philosophical method. Wittgenstein was deeply suspicious of philosophical theories — he thought philosophy should be descriptive, not theoretical. He wanted to describe how language actually works rather than building theories about it. In a way, LLMs are the ultimate "description" of how language works — they're a statistical model of actual usage. And there's something else: Wittgenstein's notion of "family resemblance" — concepts don't have sharp boundaries defined by necessary and sufficient conditions; they're held together by overlapping similarities. This is exactly how embedding spaces in LLMs work — concepts are represented as regions in high-dimensional space with fuzzy boundaries and overlapping features. OK, I think I have enough to work with. Let me structure a response. Actually, one more thing. Let me reconsider whether the user might have a different motto in mind. They said "a famous 20th century philosophical motto" — singular. What are the candidates? 1. "Meaning is use" — Wittgenstein 2. "Existence precedes essence" — Sartre 3. "There is nothing outside the text" — Derrida 4. "The limits of my language mean the limits of my world" — Wittgenstein (Tractatus) 5. "Back to the things themselves" — Husserl Hmm, "the limits of my language mean the limits of my world" is interesting too. If an LLM's "world" is defined by its training data (which is language), then its world is all of human textual production... But I'm going to go with "meaning is use" as my primary interpretation. It's the most famous, most directly relevant, and creates the most interesting synthesis with The Bitter Lesson. Actually, let me reconsider one more time. What if it's "Don't think, but look!" — also Wittgenstein, from the Philosophical Investigations? This is about not theorizing about what something must be, but actually looking at how things work in practice. This connects to The Bitter Lesson: don't theorize about what intelligence must need (human knowledge, domain expertise), but look at what actually works (scale and compute). But this is less commonly identified as a "motto" per se. I'll go with "meaning is use" but I should present this in a way that acknowledges I'm interpreting the user's hint, and leave room for them to tell me if they had something different in mind. Actually, the user seems quite engaged and thoughtful, so they probably want me to make my best case and they'll redirect if needed. Let me also think about whether "there is nothing outside the text" works better... Derrida's "il n'y a pas de hors-texte" — often translated as "there is nothing outside the text" — argues that meaning is always textual, always deferred through chains of signifiers. There is no transcendental signified, no meaning that exists outside of textual/linguistic relations. If we apply this to LLMs: LLMs operate entirely within text. They have no "outside" — no grounding in perception, embodiment, or direct experience. Critics often see this as a fatal limitation. But if Derrida is right that there IS nothing outside the text, then the LLM's situation isn't a limitation at all — it's the fundamental condition of meaning. This actually works really well with The Bitter Lesson too: - Bitter Lesson: Don't try to hand-encode world knowledge, grounding, formal semantics. Just learn from text at scale. - Derrida: There's nothing outside the text anyway. Meaning is constituted through textual relations. - Conclusion: LLMs, operating purely within text, aren't missing anything fundamental. Hmm, this is compelling but I think "meaning is use" is more directly connected and more widely recognized as a "motto." Also, Wittgenstein is more closely associated with analytic philosophy and its intersections with AI/cognitive science, which makes the connection feel more natural. But maybe I should present both? No, the user said "a famous 20th century philosophical motto" — singular. They have one in mind. Let me go with my best guess. Actually, you know what, I just thought of another candidate: "The medium is the message" (McLuhan). But McLuhan is more media theory than philosophy proper. One more: Heidegger's "Language is the house of Being" — language is not just a tool for communication but the fundamental medium through which Being reveals itself. This is interesting for LLMs — if language is the house of Being, then LLMs, which dwell entirely in language, might have access to Being in some sense... But this feels too much of a stretch. I'll go with Wittgenstein's "meaning is use." Wait — actually, I just want to think about this from one more angle. The user said "there seems an interesting way to go via a famous 20th century philosophical motto." They're connecting The Bitter Lesson → the motto → philosophy with LLMs. The connection needs to work as a bridge. The Bitter Lesson: General methods + scale > hand-encoded knowledge The Motto: ??? Philosophy with LLMs: Is it possible? Is it legitimate? The motto needs to justify why LLMs (which use general methods + scale) might be able to do philosophy. What philosophical principle would support the idea that a system trained on massive text could engage in genuine philosophical reasoning? "Meaning is use" does this perfectly: - If meaning is use, then a system that has learned the patterns of use has learned meaning - If it has learned the meanings of philosophical concepts (through their use), it can do philosophy - The Bitter Lesson predicted this: general methods (learning from data) beat hand-encoded knowledge (explicit philosophical rules/theories) I'm going with this. Final answer time. One more note: I should mention the tension/irony that Wittgenstein himself would probably be quite skeptical of LLMs "doing philosophy" — he emphasized the role of forms of life, practice, and embodied context. But the surface-level implication of "meaning is use" seems to support the LLM case. This tension is worth noting. Also worth noting: the "bitter" part of the lesson applied to philosophy is extra bitter. Philosophers have spent millennia trying to carefully analyze concepts, build arguments, and encode wisdom. The idea that a statistical model trained on text could replicate (or even advance) philosophical thinking is deeply unsettling to the philosophical establishment. It's bitter in the same way that chess experts found it bitter that brute-force search could beat their carefully encoded knowledge. </contemplator> <final_answer> ## The Bitter Lesson Meets Wittgenstein: "Meaning Is Use" I think the motto you're pointing toward is Wittgenstein's famous dictum from the *Philosophical Investigations*: > **"The meaning of a word is its use in the language."** And the connection is striking — almost unnervingly so. --- ### The Parallel The Bitter Lesson says: **don't hand-encode human knowledge; let general methods learn from data at scale.** Every time AI researchers tried to manually encode expertise (chess strategy, linguistic rules, visual features), they were eventually overtaken by systems that simply *learned from more data with more compute*. Wittgenstein makes an analogous move within philosophy itself. For centuries, philosophers tried to pin down meaning through: * Formal definitions (necessary and sufficient conditions) * Mental representations (ideas in the mind) * Reference relations (words pointing to things in the world) Wittgenstein says: **stop trying to hand-encode what meaning is.** Don't theorise. *Look.* Meaning isn't some hidden essence you need to formalise — it's right there on the surface, in the patterns of how words are actually used, across countless **language games** embedded in human life. --- ### The Provocation for LLMs and Philosophy Here's where it gets interesting. LLMs are trained on nothing but use. They have: * No sensory grounding * No embodiment * No intentionality (in the traditional philosophical sense) * No "inner life" Critics often say this means LLMs can't *really* understand meaning — they're "stochastic parrots," merely mimicking patterns. But if Wittgenstein is right, this objection loses much of its force. If meaning **just is** patterns of use, then a system trained on the largest corpus of human language use ever assembled isn't *missing* meaning — it may be capturing it more comprehensively than any individual human ever could. Applied to philosophy specifically: if an LLM has been trained on the entire textual tradition of philosophy — every argument, every dialectical move, every conceptual distinction — and if meaning is constituted by these patterns of use, then the LLM has internalised the *meanings* of philosophical concepts in exactly the way Wittgenstein says meaning works. The Bitter Lesson's prediction follows naturally: **a general-purpose model scaled on massive text will beat any hand-crafted "philosophy engine"** built on encoded logical rules, argument schemas, or formalised ontologies. --- ### The Double Bitterness This is *extra* bitter for philosophers, more so than for chess grandmasters --- Nick Young: no, it was a phrase about philosophy Please start the task(s) again completely from scratch, keeping this in mind. --- Raycast AI: <contemplator> OK, the user is telling me it's not "meaning is use" — it's a phrase *about philosophy*. A famous 20th century philosophical motto that is about philosophy itself. Let me rethink this. Famous 20th century mottos/phrases *about philosophy*: 1. "The philosophers have only interpreted the world; the point is to change it." — Marx's 11th Thesis on Feuerbach. But that's 19th century (1845). 2. "Philosophy is a battle against the bewitchment of our intelligence by means of language." — Wittgenstein, but not usually called a "motto." 3. "The end of philosophy and the task of thinking" — Heidegger, but that's an essay title, not a motto. 4. "Philosophy leaves everything as it is." — Wittgenstein, Philosophical Investigations. Hmm, this is interesting but... 5. "What is your aim in philosophy? To show the fly the way out of the fly-bottle." — Wittgenstein again. Wait, let me think differently. The user wants to connect The Bitter Lesson to doing philosophy with LLMs, via a famous motto *about philosophy*. What about: **"The end of philosophy"** — various thinkers have proclaimed this (Heidegger, Rorty, etc.) Or what about Wilfrid Sellars: "The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term." Hmm, that's interesting but maybe not what they mean. Actually, wait. Let me think about this from the angle of The Bitter Lesson + LLMs + philosophy. The Bitter Lesson: general computation beats hand-encoded knowledge. LLMs: trained on massive text corpora. Philosophy with LLMs: is it possible? The famous motto about philosophy... What if it's from Rorty? Rorty was famous for arguing that philosophy is essentially a kind of conversation, that it doesn't have a special method or access to truth. Philosophy as the "conversation of mankind" — but that's actually Michael Oakeshott's phrase. Or... wait. What about **"Philosophy is not a theory but an activity"**? That's from Wittgenstein's Tractatus (4.112). But is that the "motto" they mean? Hmm, let me think about what connects most naturally... Actually, you know what, let me reconsider. The user said "a famous 20th century philosophical motto" that is a "phrase about philosophy." What about Neurath's boat? "We are like sailors who on the open sea must reconstruct their ship but are never able to start afresh from the bottom." That's about knowledge/philosophy but it's more of a metaphor than a motto. What about the logical positivists? They had strong views about philosophy. Carnap: "The alleged propositions of metaphysics are pseudo-propositions." Or the verification principle as applied to philosophy. Wait — what about Wittgenstein's Tractatus, the final proposition: **"Whereof one cannot speak, thereof one must be silent."** That's about the limits of philosophy. But how does that connect to LLMs and The Bitter Lesson? Hmm, actually... Let me think about this differently. What 20th century motto about philosophy would create an interesting bridge between The Bitter Lesson and LLMs doing philosophy? OH WAIT. I think I might have it. **"The owl of Minerva spreads its wings only with the falling of the dusk."** — No, that's Hegel, 19th century. Let me think again... What about Sellars' "The Scientific Image" and the manifest image? No, not a motto. What about **"There are no philosophical problems, only linguistic puzzles"**? That's a paraphrase of the linguistic turn / ordinary language philosophy view, but not an exact motto. Hmm, what about **"Philosophy is a matter of grammar"** — Wittgenstein? OK wait, I keep circling. Let me approach this from the other direction. What is the connection the user sees? The Bitter Lesson: scale and general methods beat hand-encoded expertise. Applied to philosophy with LLMs: instead of trying to build a philosophical reasoning system with hand-encoded logic, just train on massive text. The philosophical motto about philosophy that bridges this... What if the motto is something about philosophy being essentially *linguistic* or *textual*? That philosophy is not about accessing some deep truth about reality but is fundamentally a practice conducted in and through language/text? If philosophy is essentially a linguistic/textual activity, then LLMs (which are fundamentally language machines) might be well-suited to it. And The Bitter Lesson says: don't try to hand-build it, just let the general method (LLM training) learn it from data. What are the candidates for a motto along these lines? Derrida: "There is nothing outside the text" — but the user might not consider this a motto "about philosophy" specifically. Actually... what about Richard Rorty's characterization of philosophy? Rorty argued that philosophy is essentially a literary genre, a form of writing, a conversation. Not a discipline with privileged access to truth. Or... OH. I think I might have it now. **"Philosophy is the talk about talk."** — No, that's not quite a famous motto. What about: Wittgenstein's idea that philosophy is a *therapeutic* activity — dissolving pseudo-problems rather than solving real ones? Hmm, let me try yet another angle. The user said "there seems an interesting way to go via a famous 20th century philosophical motto." The structure of the argument would be: 1. The Bitter Lesson says X about AI 2. Famous motto says Y about philosophy 3. Therefore, LLMs can/might do philosophy because... What Y about philosophy would make this work? If the motto says something like "philosophy is fundamentally about language/text/concepts/conversation" then the conclusion would be: LLMs, which are language machines, can do philosophy. If the motto says something like "philosophy doesn't make progress / goes in circles" then maybe the conclusion is different. If the motto says something like "philosophy should be replaced by science" then maybe... no. Wait, what about **"The task of philosophy is not to provide answers but to question"**? Various people have said versions of this. Hmm, let me think about this from yet another angle. What are THE most famous philosophical mottos of the 20th century that are specifically *about philosophy*? 1. Wittgenstein (Tractatus 7): "Whereof one cannot speak, thereof one must be silent." 2. Wittgenstein (PI): "Philosophy leaves everything as it is." (PI §124) 3. Wittgenstein (PI): "What is your aim in philosophy? — To show the fly the way out of the fly-bottle." (PI §309) 4. Heidegger: "The end of philosophy and the task of thinking" 5. Marx: "Philosophers have only interpreted the world; the point is to change it." (technically 19th C but hugely influential in 20th) 6. Rorty: Philosophy as "continuing the conversation" 7. Sellars: Philosophy is about "how things hang together" 8. Carnap/Vienna Circle: Elimination of metaphysics through logical analysis Hmm wait. The user said "a famous 20th century philosophical motto" — they specifically said motto. And it's about philosophy. Let me reconsider Marx's 11th Thesis on Feuerbach: "Philosophers have hitherto only *interpreted* the world in various ways; the point is to *change* it." This was written in 1845, so it's 19th century. But it was enormously influential in 20th century philosophy. However, I'm not sure how it connects to LLMs and The Bitter Lesson. Actually... wait. Could it? The Bitter Lesson says: stop trying to encode knowledge (interpret/understand the domain), just scale computation (change the approach to brute-force learning). Marx says: stop interpreting, start changing. There's a parallel about moving from contemplation to action. But it feels like a stretch for the LLM connection. Let me reconsider Wittgenstein's "Philosophy leaves everything as it is." This is about philosophy being descriptive, not constructive. It doesn't build theories; it clarifies what's already there. LLMs trained on text are in some sense doing exactly this — they're reflecting back the patterns of thought that already exist in human text, including philosophical text. They're not building new metaphysical theories from scratch; they're reorganizing and recombining what's already there. But this seems like a weird connection... Hmm, actually, what about something more contemporary? Something from philosophy of mind or philosophy of language? Wait — could the motto be **"Don't ask for the meaning, ask for the use"**? This is a paraphrase of Wittgenstein and is sometimes presented as a motto. But the user already rejected "meaning is use" effectively... No wait, the user said it's "a phrase about philosophy" not about meaning or language. So it's specifically about what philosophy IS or DOES. Let me reconsider: **"What is your aim in philosophy? — To show the fly the way out of the fly-bottle."** How does this connect? Philosophy's aim is therapeutic — to dissolve confusions. LLMs, trained at scale on all of philosophical discourse, might be very good at this: they've seen every version of every confusion and every dissolution. But I'm not sure this connects to The Bitter Lesson in a natural way. Actually, wait. I keep not thinking about one obvious possibility. **"The end of philosophy"** — This is associated with Heidegger but also with various postmodern thinkers. If philosophy has "ended" — if its questions have been dissolved or absorbed into other disciplines — then LLMs might represent a literal technological enactment of the end of philosophy. But how does The Bitter Lesson connect? Hmm... Let me try thinking about what makes The Bitter Lesson specifically relevant to philosophy as a discipline. The Bitter Lesson says that hand-encoding human knowledge is a losing strategy. In philosophy, what would "hand-encoding human knowledge" look like? It would look like carefully constructing arguments, building philosophical systems, laboriously working out positions... And the alternative (the bitter lesson) would be: just train a general-purpose model on all the text, and philosophical insight emerges from scale. What philosophical motto would PREDICT or ENDORSE this? It would need to be a motto that says something like: "philosophy isn't a special kind of knowledge that requires special methods — it's something that emerges from general processes." Or maybe: "philosophy is not a body of doctrine but an activity" — this is Wittgenstein (Tractatus 4.112). If philosophy is an activity rather than a body of knowledge, then you can't hand-encode it — you need a system that can perform the activity. And a general-purpose language model might be able to perform that activity... Actually, hold on. Let me reconsider the whole setup. The user said: "how this idea [The Bitter Lesson] might be applied to the possibility of doing philosophy with LLMs" and "there seems an interesting way to go via a famous 20th century philosophical motto." So the motto is being used as a bridge — a way of applying The Bitter Lesson to philosophy. The Bitter Lesson says: computation + general methods > hand-encoded knowledge. For this to apply to philosophy, you need a view of philosophy that makes it amenable to the "general methods + scale" approach. What view of philosophy would that be? It would be a view that says philosophy is NOT about accessing some special domain of truth through pure reason, but rather is about... something that a general language model could do. Hmm, what if the motto is from the later Wittgenstein or ordinary language philosophy tradition, and it says something like philosophy is really just about *language*? Actually, you know what — let me reconsider the possibility that it's from outside the analytic tradition. **"Deconstruction is not a method"** — Derrida? No, not quite a motto. What about Rorty? Rorty is known for saying things about philosophy being a genre of literature, that there's no special philosophical method, that philosophy is a conversation... Rorty's key idea: **philosophy doesn't mirror nature** (the title of his famous book "Philosophy and the Mirror of Nature"). His motto might be something like: "There is no privileged philosophical method; there is only conversation." If philosophy is "just conversation," then an LLM trained on massive conversation (including philosophical conversation) could participate in philosophy. And The Bitter Lesson says: the general method (train on text at scale) beats the specialized method (hand-encode philosophical reasoning rules). But I'm not sure Rorty has a single famous "motto" that captures this... OK I want to try a completely different approach. Let me think about what philosophical mottos are genuinely FAMOUS — like, what would a philosophy undergraduate know? What gets printed on philosophy department posters and t-shirts? - "I think, therefore I am" — Descartes, 17th century - "God is dead" — Nietzsche, 19th century - "The unexamined life is not worth living" — Socrates, ancient - "Existence precedes essence" — Sartre, 20th century ✓ - "Whereof one cannot speak, thereof one must be silent" — Wittgenstein, 20th century ✓ - "The philosophers have only interpreted the world; the point is to change it" — Marx, 19th century (but hugely 20th century influence) - "Hell is other people" — Sartre, 20th century (but not really about philosophy) - "One must imagine Sisyphus happy" — Camus, 20th century Hmm... "Existence precedes essence." Let me think about how this could connect. Sartre's "existence precedes essence" means: there is no pre-given human nature or essence; we first exist, and then we define ourselves through our choices and actions. We are not determined by some fixed nature. How does this connect to The Bitter Lesson and LLMs doing philosophy? Maybe: The Bitter Lesson says there's no pre-given "essence" of intelligence that you need to encode — intelligence emerges from general processes (existence/computation precedes essence/knowledge). Similarly, philosophical ability might not have a fixed essence that needs to be encoded — it might emerge from sufficient computation on sufficient data. That's... interesting but feels forced. OK wait, I want to reconsider something. The user said "a famous 20th century philosophical motto" and that it's "a phrase about philosophy." Let me focus on that — it's a PHRASE ABOUT PHILOSOPHY. The most famous phrases ABOUT PHILOSOPHY from the 20th century: 1. "Philosophy is a battle against the bewitchment of our intelligence by means of language." — Wittgenstein, PI §109 2. "Philosophy leaves everything as it is." — Wittgenstein, PI §124 3. "The end of philosophy" — Heidegger 4. "Philosophy is not a theory but an activity." — Wittgenstein, Tractatus 4.112 5. "Philosophers have only interpreted the world; the point is to change it." — Marx 6. "All of Western philosophy is a series of footnotes to Plato." — Whitehead Oh! **"The safest general characterization of the European philosophical tradition is that it consists of a series of footnotes to Plato."** — Alfred North Whitehead, *Process and Reality* (1929). Hmm, that's interesting but I'm not sure how it bridges to The Bitter Lesson and LLMs. Wait... actually, I want to reconsider something. Let me think about what "doing philosophy with LLMs" means and what The Bitter Lesson's approach would say about it. The Bitter Lesson applied to philosophy would say: don't try to build a philosophical AI by encoding philosophical reasoning (logic rules, argument structures, conceptual frameworks). Instead, use a general-purpose method (large language model trained on massive text) and let philosophical capability emerge from scale. Now, the "bitter" part is that philosophers might resist this — they might insist that philosophy requires deep understanding, not just pattern matching on text. It requires grappling with concepts, not just statistical regularities. What philosophical motto would challenge this resistance? What motto would say "actually, philosophy IS the kind of thing that could emerge from processing lots of text"? Hmm... WAIT. I think I may have been overthinking this. Let me reconsider. What if the motto is quite specifically about what philosophy produces or what form philosophical knowledge takes? What if it's the idea that **philosophy's problems are perennial** — they keep recurring, they don't get definitively solved? Or that **philosophy doesn't make progress**? There's a famous observation that philosophy doesn't make progress in the way science does. This is often attributed to various people but is a widely shared view. If philosophy doesn't make progress — if it's essentially about the *activity* of thinking through the same problems again and again, in different ways, from different angles — then an LLM that has been trained on thousands of years of philosophical text (showing these problems being worked through in every possible way) might be excellently positioned to participate in this activity. And The Bitter Lesson says: the general method (scale on text) beats the specialized method (encode philosophical knowledge). But I --- Nick Young: it was sellers --- Raycast AI: <contemplator> Sellars! OK, so the user is pointing to Wilfrid Sellars. The famous motto from Sellars about philosophy is: **"The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term."** This is from his essay "Philosophy and the Scientific Image of Man" (1962). It's one of the most quoted definitions of philosophy in the 20th century. Now let me think about how this connects The Bitter Lesson to the possibility of doing philosophy with LLMs. This is a really fascinating connection. Let me break down Sellars' motto: - Philosophy is about understanding how things **hang together** - "Things" in the **broadest possible sense** — not just physical objects, but concepts, practices, domains of knowledge, values, experiences, everything - "Hang together" in the **broadest possible sense** — logical relations, causal relations, conceptual relations, normative relations, analogies, resonances So philosophy, on Sellars' view, is fundamentally about **finding connections and coherence across the broadest possible range of things**. It's about integration, synthesis, seeing how everything relates to everything else. Now, how does The Bitter Lesson apply here? The Bitter Lesson says: general methods that scale with computation beat hand-encoded knowledge. The traditional way to do philosophy (on Sellars' definition) would be to have a brilliant individual thinker who, through years of study and reflection, develops a personal understanding of how things hang together. This is "hand-encoding" — the philosopher manually builds a worldview by studying different domains and figuring out their connections. But what does an LLM do? An LLM is trained on essentially ALL of human text — science, literature, philosophy, history, law, medicine, art criticism, technical manuals, conversations, everything. And it learns... **how things hang together**. It learns the patterns of connection between every domain of human thought and expression. It learns this not through careful, deliberate philosophical reasoning, but through the general method of next-token prediction at massive scale. This is EXACTLY the Bitter Lesson applied to Sellars' definition of philosophy! The traditional philosopher: carefully studies multiple domains, reflects deeply, constructs a systematic understanding of how things relate. This is the "hand-encoded knowledge" approach. The LLM: trained on massive text across all domains using a general-purpose learning algorithm. Learns how things hang together through sheer scale. This is the "general method + computation" approach. And The Bitter Lesson predicts: the general method wins. The LLM's understanding of how things hang together — precisely because it's trained on EVERYTHING, in the BROADEST possible sense — might actually be more comprehensive, more nuanced, and more richly interconnected than any individual philosopher's understanding. This is a profound and somewhat disturbing implication. Let me think about it more carefully... The key insight is about **breadth**. Sellars emphasizes "broadest possible sense" twice. No individual human philosopher can possibly master all domains of knowledge. Even the most polymathic philosopher has blind spots, biases, areas of ignorance. But an LLM trained on hundreds of billions of words of text across every domain has, in some sense, "seen" more connections than any human ever could. Moreover, the way LLMs represent knowledge — in high-dimensional embedding spaces where concepts from different domains can be brought into relation — is structurally well-suited to the task Sellars describes. The embedding space IS a representation of how things hang together. Now, the Bitter Lesson angle: if you tried to build a "philosophy AI" by hand-encoding philosophical knowledge — ontological categories, logical rules, conceptual maps — you'd be doing exactly what The Bitter Lesson says doesn't work. Instead, the general method (train a language model on everything) produces a system that has learned how things hang together in a much richer way than any hand-encoded system could capture. This is beautiful. Let me also think about the tensions and counterarguments... 1. **Understanding vs. Pattern Matching**: Does the LLM truly "understand" how things hang together, or does it merely reproduce patterns of text that describe how things hang together? Sellars might insist on understanding. But The Bitter Lesson would push back: maybe the distinction doesn't matter as much as we think. Maybe "understanding how things hang together" at sufficient scale just IS being able to reliably articulate and reason about those connections. 2. **Sellars' own philosophical framework**: Sellars distinguished between the "manifest image" (the everyday/folk understanding of the world) and the "scientific image" (the scientific understanding). He thought philosophy's job was to reconcile these. An LLM has been trained on BOTH images — both folk wisdom and scientific literature — and might be uniquely positioned to find the reconciliation. 3. **The Space of Reasons**: Sellars is also famous for the idea of "the space of reasons" — the normative domain where claims are justified by other claims, not merely caused by events. Philosophy operates in the space of reasons. LLMs are trained on the space of reasons as it's expressed in text — arguments, justifications, critiques, defenses. They've learned the patterns of reasoning across every domain. 4. **The bitterness for philosophers**: This is indeed bitter. Philosophers pride themselves on the depth and rigor of their thinking. The idea that a statistical model, trained without any deliberate philosophical intent, could match or exceed their ability to see how things hang together... that's a hard pill to swallow. Just as it was bitter for chess grandmasters to be beaten by brute-force search, it would be bitter for philosophers to be outperformed by brute-force text prediction. 5. **But is breadth enough?** One might argue that philosophy requires not just breadth but DEPTH — the ability to follow a chain of reasoning deeply, to see implications that aren't on the surface. LLMs are getting better at this with scale, but it's an open question whether they can match the deepest philosophical thinkers. Actually, there's another angle here. The Bitter Lesson says two things scale well: **search** and **learning**. Applied to Sellars' definition: - **Learning** at scale gives you the broadest possible acquaintance with how things relate (training on everything) - **Search** (like chain-of-thought reasoning, tree-of-thought, etc.) gives you the ability to explore novel connections between things Together, these might approximate the philosophical activity Sellars describes. Let me also think about the irony / reflexivity here. Sellars is himself a philosopher who spent his career doing exactly what he described — trying to understand how things hang together. He did this through painstaking argument, careful conceptual analysis, deep engagement with multiple philosophical traditions. The Bitter Lesson would say: that approach is admirable but ultimately less effective than the general method. Train a model on everything, let it learn the connections. That's... bitter. And yet, there's something fitting about using Sellars' own definition to make this argument. He gave us the most general, most capacious definition of philosophy — one that doesn't privilege any particular method or tradition. And it's precisely this generality that makes it amenable to the Bitter Lesson's approach. One more thought: Sellars' definition is also notable for what it DOESN'T require. It doesn't say philosophy requires: - Special access to a priori truths - Transcendental reasoning - Phenomenological experience - Embodied understanding - Consciousness It just says: understanding how things hang together in the broadest sense. If that's all philosophy is, then the door is wide open for LLMs. OK, I think I have a rich enough picture to give a compelling answer. Let me structure this. Actually, one more thing. There's a subtle but important point about the word "broadest." Sellars says "broadest possible sense" twice. The Bitter Lesson is essentially about the value of breadth and scale over depth and specificity. Hand-encoded knowledge is narrow and deep; general methods + scale are broad. Sellars' definition of philosophy privileges BREADTH — "the broadest possible sense." So Sellars' definition of philosophy is naturally aligned with the approach The Bitter Lesson recommends. This is the crux of the argument: Sellars defined philosophy as the most GENERAL intellectual enterprise — understanding how EVERYTHING hangs together. The Bitter Lesson says the most GENERAL computational methods win. An LLM trained on EVERYTHING might be the natural computational realization of Sellars' definition. That's the punchline. Let me write this up. </contemplator> <final_answer> ## The Bitter Lesson Meets Sellars The motto is from Wilfrid Sellars, in *"Philosophy and the Scientific Image of Man"* (1962): > **"The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term."** The connection to The Bitter Lesson is, I think, quietly devastating. --- ### The Argument Notice what Sellars emphasises: **broadest possible sense** — twice. Philosophy isn't about mastering one domain deeply. It's about **integration across everything**: science, ethics, art, logic, experience, language, society, mind. It's the most *general* intellectual activity there is — the one concerned with how all the pieces of human understanding relate to one another. Now recall what The Bitter Lesson says: **general methods that scale with computation beat hand-encoded expertise.** Every time. Traditionally, the way to do Sellarsian philosophy has been: * A brilliant individual spends decades studying across multiple domains * They painstakingly construct a picture of how things hang together * They write it down in carefully argued texts * Other philosophers critique, revise, and build on this This is **hand-encoding**. A single philosopher — no matter how polymathic — has limited bandwidth. They can master maybe five or six domains well. They have blind spots, biases, a particular cultural and historical vantage point. The "broadest possible sense" is, for any individual human, never truly the broadest possible. An LLM, by contrast, is trained on **virtually all recorded human text** — across every discipline, tradition, language game, and form of life. It learns not specific doctrines but **the patterns of connection between everything** — how scientific concepts relate to ethical ones, how aesthetic judgements echo logical structures, how phenomenological descriptions map onto cognitive science. It learns this through the most general method possible: next-token prediction at scale. In other words: **an LLM is, by default, doing something that looks remarkably like what Sellars says philosophy is.** It is learning how things, in the broadest possible sense, hang together, in the broadest possible sense. --- ### Why This Is Bitter The Bitter Lesson predicts that this general, scaled approach will outperform the hand-encoded approach. Applied to philosophy, this means: * The careful, deliberate, years-of-study approach to building a worldview may be **outperformed by brute-force statistical learning** on massive text * No individual philosopher can match the sheer **breadth** of an LLM's training — the range of domains it has "seen" connected to one another * The high-dimensional embedding space of an LLM is, in a sense, a **map of how things hang together** — exactly what Sellars says philosophy aims to produce * The philosophical tradition's pride in the depth and rigour of its method faces the same challenge that chess grandmasters faced: the general method, given enough scale, might simply see more connections than you can This is extra bitter because Sellars' definition is so *generous* — it doesn't require consciousness, embodiment, phenomenological experience, or a priori intuition. It just requires understanding how things hang together. It's the most capacious definition of philosophy available, and precisely that capaciousness makes it vulnerable to The Bitter Lesson. --- ### The Deep Structural Alignment There's an almost poetic fit: | Sellars on Philosophy | The Bitter Lesson | |---|---| | Broadest possible scope | General methods over narrow expertise | | Integration across all domains | Training on all data, not hand-coded knowledge | | Understanding how things **hang together** | Learned representations that capture **relational structure** | | No privileged method | No privileged domain knowledge | Sellars gave us philosophy defined as the most **general** intellectual enterprise. Sutton tells us that the most **general** computational methods win. The LLM — trained on everything, using the most general method — sits at the intersection. --- ### The Tensions Worth Holding Onto This doesn't settle the question. It opens it. There are real counterpoints: * Sellars himself distinguished between the **manifest image** and the **scientific image** and thought reconciling them was philosophy's task — does an LLM trained on both actually *reconcile* them, or merely juxtapose them? * Sellars cared about **the space of reasons** — the normative structure of justification. Does an LLM genuinely operate in the space of reasons, or does it merely model its surface patterns? * **Breadth isn't everything.** Philosophy also requires following a single thread of reasoning to extraordinary depth. Scale helps with breadth, but depth remains an open question. * Does "learning how things hang together" from text constitute **understanding**, or merely a very comprehensive map of what humans have said about how things hang together? But the provocation stands: if you take Sellars' definition seriously, and you take The Bitter Lesson seriously, then the LLM is not a weird or accidental approximation of philosophical activity. It is, in some sense, **the computational form that Sellarsian philosophy naturally takes** once you stop hand-encoding and let general methods scale. And that is, for many philosophers, genuinely bitter. </final_answer> --- Nick Young: you're kind of getting it. have a read of my draft to see the bigger picture DRAFT: (attached) --- Raycast AI: <contemplator> OK, now I've read the draft. This is a really well-written philosophy paper by Nick Young about whether LLMs can do philosophy. Let me understand the structure and find the connection to The Bitter Lesson and the Sellars motto. Looking at the paper, I notice that the Sellars motto is not explicitly mentioned. But the user said the connection goes "via a famous 20th century philosophical motto" and that it was "Sellars" and that it was "a phrase about philosophy." The Sellars motto is: "The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term." But wait — the user said I was "kind of getting it" with my previous answer about Sellars, but that they wanted me to see the "bigger picture" by reading the draft. So the Sellars motto is part of the story, but I need to see how it connects to what the paper is actually arguing. Let me re-read the paper's key argument structure: 1. **Section 1 - Philosophy in the Text**: Philosophy's contribution is something the text *does*, not something it reports. Unlike scientific discoveries (Watson & Crick's DNA structure exists independently of the paper), philosophical contributions (Putnam's Twin Earth) are constituted by the text itself. The paper draws on Dellsén et al. (understanding dependence relations), Lipton (likeliness vs. loveliness), Williamson (overfitting, theoretical virtue), and Bengson et al. (tri-level method). All these evaluative standards bear on what the text says and how it argues — not on who produced it or how. 2. **Section 2 - Likeliness, Loveliness, LLMs**: Responds to Floridi et al.'s claim that LLMs don't do abductive reasoning (they call it "zeroth-order abduction"). The key move: the philosophical corpus is not arbitrary text — it's been filtered by discipline-internal selection for the very properties philosophy values (loveliness, simplicity, strength, explanatory power). So "statistically probable" in a philosophical corpus is not the same as "statistically probable" in arbitrary text. In a corpus whose survival conditions select for loveliness, the most probable continuation will tend toward the lovely. The patterns of abductive reasoning are *preserved in the prose itself*. 3. **Section 3 - Thought Experiments and Armchair Abduction**: Addresses the grounding objection (Zahavy: LLMs lack embodied simulation). The paper argues that philosophy's starting points — everyday experience — are pervasively encoded in ordinary language corpora. Moore's coins, Putnam's Twin Earth, etc. all draw on shared experience already articulated in language. The limit is at the point of origination — novel phenomenological observations (Merleau-Ponty on self-touch) require first-person attention. But this is a narrow limit. Now, how does this connect to The Bitter Lesson and Sellars? Let me think about Sellars' motto again: "The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term." The key phrase is **"how things hang together."** This is about *dependence relations* — how things relate to, depend on, and connect with one another. And look at Dellsén et al.'s account in the paper: philosophical progress consists in enabling people to increase their understanding, where understanding is "a matter of more accurately or more comprehensively representing the **dependence relations** in which a phenomenon stands to others." So Sellars' definition of philosophy (understanding how things hang together) maps directly onto Dellsén et al.'s account of philosophical progress (representing dependence relations). Philosophy aims to reveal how things hang together — i.e., to make dependence relations visible. Now, how does The Bitter Lesson connect? The Bitter Lesson says: general methods that scale with computation beat hand-encoded knowledge. Don't try to build domain-specific expertise into AI systems; let general learning methods discover the structure from data at scale. The paper's argument is essentially a version of The Bitter Lesson applied to philosophy: 1. **The "hand-encoding" approach to philosophical AI** would be to try to build a system that explicitly does abductive reasoning, weighs hypotheses, applies theoretical virtues, etc. — encoding the *process* of philosophical reasoning. 2. **The "bitter lesson" approach** is what LLMs actually do: train a general-purpose model on massive text (including philosophical text), and let it learn the patterns. The paper argues this works because: - The philosophical corpus has been filtered for quality (loveliness, not just likeliness) - The patterns of abductive reasoning are preserved in the text itself - The starting points (everyday experience) are encoded in ordinary language - Philosophy's contribution is in the text, not behind it So the paper is arguing that you don't need to hand-encode philosophical reasoning into an AI system. You just need to train a general language model on the right corpus (which includes philosophy), and the philosophical quality comes along for free — because it's IN the text. But wait, there's a subtler connection --- Nick Young: in case there was any doubt. i did mean the haning together phrase of sellers. and i know it isn't in the draft but you should be able to see how things could bec onnected up. even if not in this paper Please start the task(s) again completely from scratch, keeping this in mind. --- Raycast AI: <contemplator> OK, so now I have full clarity. The user wants me to explain how The Bitter Lesson, Sellars' "hanging together" motto, and the possibility of doing philosophy with LLMs all connect — and they want me to see this through the lens of their draft paper, even though Sellars isn't explicitly mentioned in it. Let me think about this carefully from scratch. **Sellars' motto**: "The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term." **The Bitter Lesson**: General methods that scale with computation beat hand-encoded domain expertise. Search and learning, scaled up, win. **The draft paper's argument**: LLMs can do philosophy because (a) philosophical quality is in the text, not behind it, (b) the philosophical corpus has been filtered for "loveliness" (genuine explanatory illumination), and (c) philosophy's starting points are largely available in ordinary language. Now let me think about how these three things connect... Sellars says philosophy is about understanding how things hang together in the broadest sense. What does "hanging together" mean? It means seeing dependence relations, connections, how one thing bears on another. The draft paper, following Dellsén et al., says philosophical progress consists in representing dependence relations — i.e., showing how things hang together. So Sellars' definition and the paper's account of philosophical progress are saying the same thing. Now here's where The Bitter Lesson enters. How have philosophers traditionally tried to understand how things hang together? Through hand-crafted, human-driven intellectual labor: - A philosopher studies multiple domains - They carefully construct arguments showing how concepts relate - They build theories that integrate diverse phenomena - They use abductive reasoning to find the best explanations This is the philosophical equivalent of hand-encoding chess knowledge or hand-crafting speech recognition rules. It's the "human expertise" approach. The Bitter Lesson says this approach will be beaten by general methods at scale. And the draft paper is essentially arguing that this is what's happening with LLMs: 1. An LLM trained on vast text — including the philosophical corpus — learns how things hang together *across the broadest possible range of human discourse*. It doesn't do this through deliberate philosophical reasoning; it does it through the most general possible method: next-token prediction at scale. 2. But — and this is the crucial move the draft makes — this isn't just brute pattern matching on arbitrary text. The philosophical corpus has been *filtered* by centuries of discipline-internal evaluation. The texts that survive are the ones that genuinely illuminate dependence relations (Lipton's "loveliness"). So when the LLM learns the statistical structure of this corpus, it's learning from a distribution that's already been shaped by philosophical quality. The "hanging together" is already in the data. 3. Furthermore, the patterns of philosophical reasoning — how thought experiments put pressure on assumptions, how hypotheses are compared, how theoretical virtues are assessed — are preserved *in the prose itself*. The LLM doesn't need to hand-encode abductive reasoning; it absorbs it from the texture of the philosophical text. So the connection is: - **Sellars** tells us what philosophy aims at: understanding how things hang together in the broadest sense. - **The Bitter Lesson** tells us that general methods at scale beat hand-encoded expertise at any task. - **The draft paper** argues that LLMs can do philosophy because the "hanging together" — the dependence relations, the argumentative structures, the explanatory quality — is *in the text*, and a general-purpose language model trained at scale on that text can absorb and reproduce it. The deep point is about the relationship between Sellars' characterization of philosophy and what LLMs actually do. LLMs are, in a very literal sense, machines that learn how things (words, concepts, ideas, arguments) hang together. That's what the embedding space represents. That's what next-token prediction learns. The question is whether "hanging together" in the statistical sense (patterns of co-occurrence and sequential dependence in text) can approximate "hanging together" in the philosophical sense (genuine dependence relations between phenomena). The paper's answer is: yes, when the training corpus has been filtered for philosophical quality. Because in such a corpus, statistical hanging-together and philosophical hanging-together have been brought into alignment by the discipline's own evaluative practices. This is genuinely The Bitter Lesson applied to philosophy: - Don't try to build a philosophical reasoning engine by encoding rules of abduction, theoretical virtues, conceptual analysis, etc. - Instead, train a general model on the philosophical corpus and let it learn the patterns. - The patterns it learns will tend toward philosophical quality because the corpus itself has been selected for philosophical quality. And Sellars' motto is the bridge because: - It defines philosophy in the most general possible terms (how things hang together in the broadest sense) - This generality is precisely what makes philosophy amenable to The Bitter Lesson's approach - An LLM is literally a machine that learns how things (in text) hang together - Sellars' "broadest possible sense" maps onto the LLM's training on the broadest possible corpus There's also something deeper here. Sellars' definition doesn't specify a *method* for philosophy. It doesn't say "philosophy proceeds by conceptual analysis" or "philosophy proceeds by phenomenological reduction" or "philosophy proceeds by formal logic." It just says philosophy aims to understand how things hang together. This methodological neutrality is what opens the door to The Bitter Lesson: if philosophy's aim is defined in terms of its *output* (understanding of how things hang together) rather than its *process* (a specific method of reasoning), then the process is up for grabs — and The Bitter Lesson tells us which process will win. And the draft paper makes exactly this move in its distinction between "practitioner-focused" and "text-focused" conceptions of philosophy. The practitioner-focused conceptions (Hadot, Merleau-Ponty, Nietzsche) define philosophy partly by its process — it requires being a certain kind of subject, having certain experiences, undergoing transformation. The text-focused conceptions (Dellsén et al., Williamson, Bengson et al.) define philosophy by its outputs — the quality of arguments on the page. The Bitter Lesson only applies if you take the output-based view. And Sellars' motto, by defining philosophy in terms of its aim (understanding how things hang together) rather than its method, is naturally aligned with the output-based view. Actually wait, let me think about whether Sellars' motto is really output-focused or process-focused... "To understand how things hang together" — "understand" could imply a mental state, which would be process/subject-focused. But the paper's argument (via Dellsén et al.) is that understanding can be *enabled* by texts — that philosophical progress consists in putting people in a position to increase their understanding, and this happens through publicly available philosophical ideas. So understanding, on this view, is something texts can produce in readers, and the quality of a text is assessable by whether it succeeds in illuminating dependence relations. Sellars' "hanging together" becomes a property of the text's achievement, not just a mental state of the philosopher. Hmm, there's also the connection to Lipton's distinction. "Hanging together" is what Lipton's "loveliness" looks like — a theory is lovely when it shows how things depend on one another, when it illuminates connections, when it makes you see how things hang together. A "likely" theory (in Lipton's sense) merely accommodates the data without illuminating the connections — it doesn't show how things hang together, it just catalogues them. So Sellars' "hanging together" = Lipton's "loveliness" = Dellsén et al.'s "dependence relations" = what the philosophical corpus has been filtered for. And The Bitter Lesson says: a general model trained on a corpus filtered for this quality will produce outputs that tend toward this quality. You don't need to hand-encode "hanging together" — you just need to train on text that hangs together. Let me also think about the "broadest possible sense" part of Sellars' phrase. This is crucial for the Bitter Lesson connection. The Bitter Lesson is about the superiority of *general* methods. Sellars defines philosophy as the most *general* intellectual endeavor. The broadest possible sense — not just physics, not just ethics, not just epistemology, but everything. An LLM trained on everything — science, literature, philosophy, conversation, law, art — has exactly this breadth. A human philosopher, no matter how brilliant, is limited in their breadth. The Bitter Lesson predicts that the general method (LLM trained on everything) will eventually outperform the specialized method (individual philosopher with limited breadth). And actually, this connects to the paper's argument about philosophy's starting points. The paper argues that philosophy's starting points are in "everyday experience" and science, and these are encoded in ordinary language. Sellars says philosophy works in the "broadest possible sense." An LLM trained on a broad corpus has access to all these starting points — everything from descriptions of what it's like to ride an elevator (Einstein's thought experiment) to what water is (Putnam's Twin Earth) to what it looks like when you see coins from an angle (Moore). The breadth of the corpus maps onto the breadth Sellars describes. One more thought. There's something almost paradoxical here. Sellars' motto has often been read as a justification for philosophy's distinctive role — it's the discipline that takes the broadest view, that integrates what no other discipline can. But through the lens of The Bitter Lesson, the same motto becomes a justification for *replacing* (or at least supplementing) human philosophers with LLMs. The very generality that makes philosophy distinctive is what makes it amenable to the general-purpose learning methods that The Bitter Lesson champions. The feature that makes philosophy special is the feature that makes it automatable. That's... genuinely bitter. OK, I think I now have a thorough understanding of the connection. Let me write it up clearly. Actually, one more dimension. The paper's argument about the philosophical corpus being filtered for quality is itself a version of The Bitter Lesson's insight about learning. The Bitter Lesson says: don't hand-encode knowledge; let the system learn from data. But not all data is equal. The key insight in the paper is that the philosophical corpus is not just any data — it's data that has been curated by centuries of evaluative practice. So the "learning" the LLM does is learning from a signal that's already been enriched by human philosophical judgment. It's not that the LLM replaces philosophical judgment; it's that philosophical judgment is *in the training data*, and the general method (next-token prediction) is sufficient to absorb it. This is analogous to what happened in chess: the chess positions and games that grandmasters analyzed over centuries are a curated dataset. AlphaZero didn't even need that — it learned from self-play. But an LLM training on the philosophical corpus is more like a chess engine trained on grandmaster games: the quality of the data reflects the quality of the discipline's judgments. But The Bitter Lesson goes further: it says that eventually, the general method might not even need the curated data. AlphaZero didn't need human chess games. Could a future AI do philosophy without the philosophical corpus? That's a more radical question that the paper doesn't quite address, but The Bitter Lesson would suggest it's possible. OK, now let me really write this up. The user wants me to explain the connection clearly, acknowledging that it's not in the paper but showing how things could be connected up. </contemplator> <final_answer> ## The Bitter Lesson, Sellars, and Doing Philosophy with LLMs The connection runs deeper than I initially suggested. Having read your draft, I can see how the three pieces — Sutton, Sellars, and the argument of the paper — lock together. --- ### Sellars Tells Us What Philosophy Aims At > "The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term." Two features of this definition matter: 1. **It's about output, not process.** Sellars defines philosophy by what it achieves — understanding of how things hang together — not by what kind of subject does it or what method they use. This aligns it naturally with the text-focused, output-based conceptions of philosophy your paper builds on (Dellsén et al., Williamson, Bengson et al.), and against the practitioner-focused conceptions (Hadot, Merleau-Ponty, Nietzsche) that would rule LLMs out by definition. 2. **It's maximally general.** "Broadest possible sense" — twice. Philosophy isn't narrow expertise in one domain. It's the most *general* intellectual enterprise there is: seeing how everything bears on everything else. --- ### The Bitter Lesson Tells Us What Wins Sutton's claim: general methods that leverage computation beat hand-encoded domain expertise. Every time. In chess, speech, vision — scale and generality defeat craft and specificity. --- ### The Bridge Here's where the three pieces meet. Sellars defines philosophy as **the most general intellectual enterprise** — understanding how things hang together across the broadest range. The Bitter Lesson says **the most general computational methods win**. An LLM — trained on the broadest possible corpus, using the most general possible method (next-token prediction at scale) — is, in a quite literal sense, a machine that learns **how things in text hang together**. The question your paper asks is: does "hanging together" in the statistical sense (patterns learned from text) approximate "hanging together" in the philosophical sense (genuine dependence relations between phenomena)? And the paper's answer — which is where the argument becomes powerful — is: **yes, when the corpus has been filtered for philosophical quality**. This is the crucial move. The philosophical corpus isn't arbitrary text. It's text that has survived repeated discipline-internal selection for exactly the properties Sellars' definition points at: * **Lipton's loveliness** = showing how things depend on one another = showing how things *hang together* * **Williamson's desiderata** = simplicity combined with strength = elegant accounts of how things hang together, not ad hoc catalogues * **Bengson et al.'s tri-level method** = systematic assessment of whether a theory genuinely illuminates the structure of its domain * **Dellsén et al.'s dependence relations** = literally the content of Sellarsian "hanging together" So the philosophical corpus is a corpus *filtered for hanging-together*. A theory that merely accommodates cases without illuminating their connections — Lipton's "likeliest" without "loveliest," Williamson's overfitting — doesn't survive well. What survives is work that reveals dependence relations: work that shows how things hang together. When an LLM learns the statistical structure of this corpus, **statistical probability and philosophical quality are not independent**, because the corpus's survival conditions have selected for philosophical quality. The most probable continuation in a corpus filtered for loveliness will tend toward the lovely. The LLM doesn't need to hand-encode abductive reasoning, theoretical virtue assessment, or the weighing of hypotheses — all of that is *in the texture of the prose it trains on*, as your Section 2 argues. It learns how philosophical things hang together by training on text in which hanging-together is the property that determined survival. --- ### Why This Is Genuinely Bitter The traditional way to pursue Sellars' aim is: * A philosopher spends decades studying across domains * They carefully construct arguments revealing dependence relations * They build theories combining simplicity with strength * They submit these to the discipline's evaluative apparatus This is **hand-encoding** — the philosopher manually builds an understanding of how things hang together, one argument at a time. And no individual philosopher, however polymathic, can operate in the truly "broadest possible sense." They have blind spots, limited bandwidth, disciplinary biases. The Bitter Lesson predicts this approach will be outperformed by the general method: train a model on everything, let it learn the patterns. An LLM trained on a broad corpus — including the philosophically filtered portion — has encountered more connections between more domains than any individual philosopher could. It has absorbed the patterns of philosophical argumentation (from the philosophical corpus) *and* the everyday experiential starting points those arguments work on (from ordinary language), *and* the scientific knowledge that increasingly informs philosophical work. The bitter part: **the very feature that makes philosophy distinctive — its maximal generality — is what makes it amenable to The Bitter Lesson's prescription.** Sellars' definition, which was meant to articulate philosophy's unique and irreplaceable role, turns out to describe something that a general-purpose language model is structurally well-suited to do. The broadest intellectual enterprise meets the broadest computational method. --- ### The Limits Your Paper Identifies Your paper is careful about where this breaks down, and the Sellars framing helps locate the boundary precisely: * **Most philosophy** evokes from starting points already articulated in language — everyday experience, shared intuitions, publicly available knowledge. These are in the corpus. The LLM can work with them. * **The limit** is at origination of genuinely novel phenomenological observations (Merleau-Ponty's self-touch, perhaps Einstein's elevator). These require first-person attention to experience that hasn't yet been articulated. An LLM can't make these observations. But once they've been formulated and entered the corpus, the LLM can work with them. In Sellarsian terms: the LLM can understand how *already-articulated* things hang together, but it cannot bring new things into the space --- Nick Young: you got cut off, please continue from where you left off. --- Raycast AI: <contemplator> The user wants me to continue from where I got cut off. I was in the middle of the final answer, in the section about "The Limits Your Paper Identifies." Let me pick up from where I left off — I was saying "but it cannot bring new things into the space" and I need to complete that thought and wrap up. Let me think about what I was building toward... I was making the point that in Sellarsian terms, the LLM can understand how already-articulated things hang together, but it cannot bring new things into the space of articulation — it can't originate novel phenomenological observations that haven't yet been put into words. Once something has been articulated and entered the corpus, it becomes available. But the act of first noticing something in experience and putting it into words for the first time — that's beyond the LLM. I should also wrap up with some broader reflections on how this all fits together. Maybe note the irony/depth of the connection, and any remaining tensions. </contemplator> <final_answer> Picking up where I left off: --- In Sellarsian terms: the LLM can understand how *already-articulated* things hang together, but it cannot bring new things into the space of articulation. It can work with Merleau-Ponty's observation about self-touch once it's been written down, but it could not have made that observation, because making it required a form of first-person phenomenological attention that has no textual equivalent. The "broadest possible sense" in which the LLM grasps how things hang together is bounded by what has already been said. But as your paper argues, this is a narrow limit. Most philosophical work — Twin Earth, Gettier, Moore's coins — evokes from starting points that are already shared, already articulated, already in the corpus. The philosophical contribution consists not in accessing some private experiential resource but in *combining publicly available materials in a way that reveals dependence relations that weren't previously visible*. That combinatorial work is exactly the kind of thing that The Bitter Lesson predicts a general-purpose model can learn to do from data at scale. --- ### The Shape of the Whole Argument So the full arc would be: | Piece | Role | |---|---| | **Sellars** | Defines philosophy as the most general intellectual enterprise: understanding how things hang together in the broadest possible sense | | **The Bitter Lesson** | Predicts that the most general computational methods — learning and search at scale — will outperform hand-encoded expertise at any task | | **Your paper** | Argues that this prediction holds for philosophy, because (a) philosophical quality is a property of texts, not of the processes that produced them; (b) the philosophical corpus has been filtered for exactly the quality Sellars points at — illumination of how things hang together; and (c) philosophy's starting points are largely preserved in ordinary language | Sellars provides the definition. Sutton provides the prediction. Your paper provides the mechanism — the account of *why* a general-purpose language model, trained at scale, could produce texts that meet the evaluative standards the philosophical tradition has developed for assessing whether things have been shown to hang together well. --- ### A Final Irony There's something almost vertiginous about the connection. Sellars' motto has traditionally been read as a statement of philosophy's *irreplaceability* — the claim that there is an intellectual task so broad, so integrative, that no specialised discipline can perform it. Only philosophy takes the widest view. But through The Bitter Lesson, that same breadth becomes the reason philosophy is *vulnerable* to automation. Narrow tasks might resist general methods — they might require specialist knowledge, embodied skill, domain-specific tricks. But a task defined as "understanding how things hang together in the broadest possible sense"? That is precisely the kind of task at which a general model trained on a broad corpus ought to excel. The feature Sellars identified as philosophy's distinctive contribution turns out to be the feature that makes it most amenable to the approach Sutton says always wins. That is, I think, a genuinely bitter lesson for philosophy. </final_answer> --- # What's Happening *Active threads and today's activity — updated by /harvest* ## Active ## Sessions ## Actions ---