# opus[1m] ## Retrieval Notes - Session id: `baac976d-159e-40f3-86a9-c037ef5a443a` - Last activity: `2026-03-30T20:05:37.509Z` ## My Notes <!-- Add your notes here. This section is preserved across syncs. --> ## Conversation ### User opus[1m] ### User I have been thinking about the ideas in my generating philosophy paper and how they relate to the bitter lesson? can you see the interesting possibilities that I do? make sure that you explain your analysis, justification, reasons etc etc. for your answer BEFORE giving me your final answer. Please give me a long, deep, and detailed answer, with lots of options/ideas: as we are in the 'kicking ideas around' stage, I want as much feedback as you can give me so that i have a lot to consider. ### Assistant I need to read your Generating Philosophy project materials before I can engage with this properly. Let me load the session file and manuscript. ### Assistant Let me read the manuscript sections to understand the full argument. ### Assistant Now let me contemplate this properly. I have the full manuscript and session context. Let me start with what the Bitter Lesson actually says, and then work through the connections systematically. Sutton's claim, stripped down: over 70 years of AI research, the approaches that won were always the ones that leveraged general-purpose computation — search and learning — over approaches that tried to hand-code human domain knowledge. Chess, Go, speech recognition, computer vision — every time, the "dumb" statistical methods that scaled with compute beat the "smart" methods that tried to build in what humans know about the domain. The lesson is "bitter" because it means our carefully accumulated expertise matters less than we'd like to think. The thing that works is not encoding what we know, but letting the system learn from data at scale. Now, Nick's paper. The argument, at its most compressed: philosophy should be evaluated by text-internal properties (Williamson's intrinsic virtues); the philosophical corpus is filtered for these virtues through peer review, citation, teaching; an LLM trained on this corpus has therefore absorbed the distribution of virtue-exhibiting text; statistical plausibility over this corpus converges with philosophical quality; the experiential inputs philosophy needs are mostly already linguistically articulated; the limit is at origination of genuinely new phenomenological observations. Where do these intersect? Hmm. Let me think about this from several angles. First angle: the Bitter Lesson as vindication of Nick's text-internal approach. The Bitter Lesson says: don't try to build in domain knowledge; let the system learn from data. Nick's paper says: the philosophical corpus IS the data, and it's already filtered for quality. These fit together remarkably well. The Bitter Lesson would predict that trying to build a "philosophical reasoning engine" — one that explicitly encodes rules of abduction, argumentation schemes, theoretical virtues — would lose to a system that just learns from a massive corpus of philosophical text. And that's... roughly what has happened? Nobody built a philosophy-specific reasoning system. GPT and Claude learned from the general corpus, which includes philosophy, and they produce philosophical text that (Nick argues) exhibits intrinsic virtues. The Bitter Lesson predicted this shape of outcome. But wait — there's a subtlety here. The Bitter Lesson is usually read as being about the futility of encoding human knowledge INTO systems. Nick's paper is about something different: the knowledge is in the corpus, and the corpus is the training data. The system doesn't need human knowledge encoded into it because the data already encodes it. This is actually a deeper point than Sutton makes. Sutton frames it as "general methods beat domain-specific methods." Nick is saying something more like: "the domain is already in the data, because philosophy IS text, and the text IS the training data." The convergence isn't accidental — it's constitutive of what philosophy is. Second angle: the virtue-filtered corpus as the mechanism that makes the Bitter Lesson work for philosophy specifically. The Bitter Lesson is a general empirical claim. It doesn't explain WHY general methods work — it just observes that they do. Nick's paper actually provides a mechanism for why the Bitter Lesson holds in philosophy's case. The mechanism is the multi-layered filtering: peer review selects for handling objections, citation selects for usefulness, teaching selects for clarity, anthologizing selects for depth. This filtering has shaped the training data so that statistical plausibility (what the model has learned to produce) converges with philosophical quality. So Nick's paper isn't just an instance of the Bitter Lesson — it explains why the Bitter Lesson works here. This is interesting because Sutton doesn't have a good explanation for WHY general methods win. He just observes that they do. Nick could be offering a domain-specific explanation: in philosophy, general methods win because the domain's own quality-control processes have pre-shaped the data in ways that make statistical learning effective. The "bitterness" of Sutton's lesson — the fact that our expertise doesn't help — has a specific explanation in philosophy: the expertise is already in the corpus. Third angle: the Bitter Lesson as an objection to Nick's paper. Wait, this is interesting too. The Bitter Lesson could be read as cutting AGAINST some of what Nick argues. Nick emphasizes the importance of prompting skill — dialectical framing, solution-gestured prompting, conversational iteration. He says "the prompter's skill consists in writing text whose good continuation is also good philosophy." But the Bitter Lesson would predict that as models get bigger and better, the need for skilled prompting will diminish. The general method (more compute, more data, better learning) will make up for what skilled prompting currently provides. If the Bitter Lesson holds, then the prompting taxonomy Nick develops in Section 4 is a temporary artifact — useful now, but destined to be made redundant by scale. This could go either way. On one hand, it's an objection to spending too much time on prompting. On the other hand, Nick could embrace it: "even the Bitter Lesson predicts that LLMs will get better at philosophy over time, not worse, because more compute and more data will only deepen the convergence between statistical plausibility and philosophical quality." Fourth angle: the Bitter Lesson and the phenomenological limit. This is where it gets really interesting. Nick's paper identifies a genuine limit: "if there is a phenomenological observation that nobody has yet made... an LLM cannot make it." Merleau-Ponty's observation about self-touch required first-person phenomenological attention. The Bitter Lesson would have something to say about this limit. Sutton's point is that people always think they've identified the thing that AI can't do — the irreducible human contribution — and they're always wrong. The researchers who thought chess required "strategic understanding" were wrong. The researchers who thought Go required "intuition" were wrong. So the Bitter Lesson might predict that even the phenomenological limit Nick identifies will eventually fall. How? Not by LLMs developing phenomenology, but by the data expanding — more human descriptions of experience, more first-person reports, richer linguistic articulations of phenomenological observations. The "pre-propositional" turns out, over enough data, to have been articulated somewhere. Or multimodal models trained on video, audio, and embodied interaction might acquire something functionally equivalent to the experiential grounding Zahavy demands. But Nick could push back: the phenomenological limit isn't like chess or Go. It's not a limit of computational difficulty. It's a limit of data availability — you can't learn from data that doesn't exist yet. No amount of compute will generate Merleau-Ponty's observation about self-touch from a corpus that doesn't contain it. The Bitter Lesson says "more compute beats hand-coding." It doesn't say "more compute beats the absence of data." Hmm, but actually, this is where it gets complicated. The Bitter Lesson might predict that the relevant "data" will come from unexpected sources — not from philosophy papers but from descriptions of embodied experience scattered across the general corpus. And Nick's own paper acknowledges something like this in the Einstein elevator example: "People describe feeling heavier when a lift accelerates and lighter as it decelerates — descriptions that presuppose no physics but encode the phenomenal contrast between inertial and gravitational force." So the Bitter Lesson might say: the phenomenological limit is real for a system trained only on philosophy, but for a system trained on the general corpus (which includes billions of first-person descriptions of embodied experience), the limit may be narrower than you think. And this connects to the Sellars quote Nick uses: philosophy is about "how things in the broadest possible sense hang together in the broadest possible sense." A system trained on everything is trained on philosophy's subject matter. Fifth angle: the Bitter Lesson and the self-proving nature of the paper. Nick's paper argues that it is itself an instance of what it describes — "submitted for blind review, evaluated by the very criteria it articulates." This is already a provocative move. But connecting it to the Bitter Lesson adds another dimension. The Bitter Lesson says that AI researchers' intuitions about what requires "real" intelligence are unreliable. Nick's paper doesn't just argue that LLMs can do philosophy — it demonstrates it, by being a product of human-LLM collaboration. And if the Bitter Lesson holds, this is just the beginning. Future collaborations will be even more effective, not because the prompting gets better, but because the models get better at leveraging the data they've already absorbed. Sixth angle: the really provocative possibility — the Bitter Lesson AS a philosophical argument. Here's where I think the deepest connection lies, and I need to be careful because this might be my own speculation rather than something Nick has already seen. The Bitter Lesson is usually read as an empirical generalization about AI research. But it can also be read as a claim about the nature of knowledge domains. Sutton's observation is that domain expertise is less important than we think — that the "bitter" truth is that general learning methods capture what matters about a domain better than expert-designed systems do. This is an epistemological claim about the relationship between explicit knowledge and the knowledge implicit in data. Nick's paper makes a structurally parallel claim about philosophy: the philosophical corpus encodes more than any individual philosopher knows explicitly. The filtering process has encoded evaluative standards that no individual philosopher could fully articulate. When Nick writes "the virtues are latent in the model: implicit in the statistical regularities, recoverable from outputs, but not explicitly represented as rules" — this is the Bitter Lesson applied to philosophical knowledge itself. And this connects to a deeper metaphilosophical point. If the Bitter Lesson is right about philosophy, it suggests that philosophical knowledge is more like empirical knowledge than philosophers have traditionally assumed. The traditional picture is that philosophical knowledge is the product of rational insight — of seeing that something must be so. But the Bitter Lesson picture is that philosophical knowledge is the product of a process (the historical evolution of the philosophical corpus) that works by filtering for quality at scale, and that this process knows more than any individual participant in it. The tradition's accumulated wisdom, encoded in statistical regularities, outperforms any individual's rational insight. This is... actually a kind of philosophical Darwinism. The corpus evolves; good arguments survive; the model inherits the evolved corpus. Seventh angle: the Bitter Lesson and the role of the prompter. Actually, let me revisit the prompting question because there's something more subtle here. The Bitter Lesson says general methods beat hand-coding. But Nick's prompting taxonomy isn't hand-coding — it's more like... steering. The prompter doesn't encode philosophical knowledge into the system; the prompter activates regions of the distribution that already encode it. This is actually compatible with the Bitter Lesson. Sutton didn't say that all human input is useless — he said that encoding domain knowledge into the system's architecture is futile. Prompting is different: it's providing context, not engineering architecture. It's like choosing which problem to give to the system, not how the system should solve it. But the Bitter Lesson might still predict that even the steering becomes less necessary. As models get better, they might need less dialectical framing to produce good philosophy — the default output might itself exhibit higher quality. The "generic, hedging text" that Nick describes as the unprompted default might be a temporary artifact of current model capability, not a permanent feature. Eighth angle: the Bitter Lesson and the relationship between philosophy and science. This is interesting. The Bitter Lesson originally applied to AI tasks that mirror scientific/engineering domains: chess, Go, image recognition, speech recognition. In each case, the "general method" (learning from data) beat the "expert knowledge" method. Nick's paper argues that philosophy is distinctively well-suited for LLMs because it IS text — the philosophical contribution is something the text does, not something it reports. The Bitter Lesson would suggest that this text-internal character makes philosophy MORE amenable to the general learning approach than even science is. In science, you need external validation (experiments, observations). In philosophy, by Nick's argument, you don't — the text is self-sufficient. So the Bitter Lesson should apply MORE strongly to philosophy than to science. This is a prediction Nick could make: if the Bitter Lesson holds, philosophy should be one of the domains where LLMs are MOST effective, not least — precisely because the text-internal character of philosophical evaluation means there's no external validation step that requires embodied experience (with the phenomenological limit as the narrow exception). The data IS the domain. The Bitter Lesson says "let the system learn from data." Philosophy's data is its own product. The circle closes. Ninth angle: could the Bitter Lesson help with the Zahavy response? Zahavy argues that scientific breakthroughs require "manipulative abduction" — embodied simulation. Nick responds by arguing that philosophy's starting points are mostly propositional and linguistically available. But the Bitter Lesson offers a different kind of response. Zahavy's argument assumes that the E→A jump (from experience to axioms) requires a particular kind of process — embodied simulation. The Bitter Lesson says: don't assume you know what process is required. The researchers who thought chess required strategic understanding were wrong — the process that worked was brute-force search. The researchers who thought Go required intuition were wrong — the process that worked was self-play reinforcement learning. So the Bitter Lesson would say to Zahavy: you're making the same mistake as the chess researchers. You think theoretical innovation requires embodied simulation because that's how Einstein did it. But that doesn't mean it's the only way. General learning methods might achieve the same outcome through a completely different process. Einstein needed to imagine the elevator because he was a human with limited computational resources. A system with access to all of physics might not need the imaginative shortcut. This is actually a pretty powerful move — it reframes Zahavy's argument as an instance of the recurring error that the Bitter Lesson identifies. "You think you know what's required, but the history of AI says you're probably wrong about that." Tenth angle: the Bitter Lesson and the "Two Empirical Questions" at the end of Section 4. Nick asks: (1) Can a general-distribution LLM produce texts exhibiting intrinsic virtues? (2) Would specialist training on philosophical texts improve performance? The Bitter Lesson would predict that the answer to (1) is yes and the answer to (2) is "less than you'd expect" — because the general method (broad training) captures something that specialist training might actually miss. Nick already gestures at this with the Sellars quote about "how things hang together in the broadest possible sense." The Bitter Lesson would strengthen this: specialist training might actually HURT, because it narrows the distribution away from the connections that philosophy draws on. This connects to an intriguing empirical prediction: a model trained specifically on philosophy might be WORSE at philosophy than a general-purpose model, because philosophical argumentation draws on the full breadth of human knowledge. The Bitter Lesson would predict this, and Nick's own framework explains why. Eleventh angle: wait, is there a tension? Let me check whether the Bitter Lesson actually conflicts with anything in Nick's paper. The Bitter Lesson says: general methods beat domain-specific methods. Nick's paper argues that prompting skill matters — that you need to know how to frame philosophical questions dialectically to get good outputs. Is this a form of domain-specific method that the Bitter Lesson would predict is temporary? I think the answer is: yes, but that's not a problem for Nick's paper. Nick could say: "Right now, prompting skill matters. The Bitter Lesson predicts it will matter less over time. That's fine — it strengthens my argument, because it means LLMs will only get better at philosophy as they scale." The prompting taxonomy in Section 4 is a description of current practice, not a permanent constraint. And the Bitter Lesson helps explain why the practice works NOW even if it will become less necessary LATER. Twelfth angle: the Bitter Lesson as a structuring principle for the paper itself. Could the Bitter Lesson provide a framing for the entire paper? Something like: "The bitter lesson of AI research is that general methods leveraging computation beat domain-specific methods encoding human knowledge. We argue that this lesson applies to philosophy as well — and that it applies MORE strongly to philosophy than to other domains, because philosophy's evaluative standards are already encoded in its textual corpus." This could be a powerful way to motivate the paper. Instead of starting from "can LLMs do philosophy?" you start from "the Bitter Lesson predicts that LLMs should be good at philosophy, and we explain why." Actually, hmm, I'm not sure this works as a framing for the whole paper, because Nick's paper also identifies genuine limits (the phenomenological boundary). The Bitter Lesson is maximally bold — it says general methods ALWAYS win. Nick's paper is more nuanced. So the relationship might be: the Bitter Lesson is the bold claim; Nick's paper is the careful philosophical analysis of where and why it holds, and where it doesn't. Thirteenth angle: the deeper metaphilosophical implications. If the Bitter Lesson applies to philosophy, what does that say about what philosophy IS? It suggests that philosophy is a domain where accumulated practice — the corpus, the tradition, the filtering — matters more than individual genius. The Bitter Lesson reframes the philosophical tradition from a series of great minds to a data-generating process whose outputs encode more than any individual contributor intended. This is a deflationary picture of philosophical genius but an inflationary picture of the tradition. And there's something self-undermining about the Bitter Lesson applied to philosophy. If the Bitter Lesson is itself a philosophical claim (an induction over the history of AI), and if LLMs trained on the general corpus can produce the Bitter Lesson as a plausible continuation, then the Bitter Lesson is evidence for its own application to philosophy. The loop closes. Fourteenth angle: what about the flip side — what the paper says about the Bitter Lesson? Nick's paper doesn't just receive illumination from the Bitter Lesson; it potentially enriches it. The Bitter Lesson is an empirical observation without a good theoretical explanation. Nick's framework — the virtue-filtered corpus, the convergence of statistical plausibility with domain-specific quality — provides one. The Bitter Lesson works because data-generating processes in mature domains have already filtered for quality. The statistical regularities in the data are not noise; they are the sedimented product of centuries of expert evaluation. General methods work because the data has done the expert's job already. This is actually a non-trivial contribution to the debate around the Bitter Lesson. Sutton's critics often argue that domain knowledge IS necessary — that general methods work only because researchers have built in domain knowledge via architecture, training procedures, etc. Nick's framework suggests a different answer: general methods work because the DATA already contains the domain knowledge, encoded by the domain's own quality-control processes. Fifteenth angle: let me think about whether there are PROBLEMS with drawing this connection. One issue: the Bitter Lesson is about AI systems improving over time as they get more compute. Nick's paper is about what current systems can do. There's a tension between "the Bitter Lesson predicts LLMs will keep getting better" and "here is a careful analysis of what current systems can and cannot do." The Bitter Lesson is future-oriented; Nick's paper is present-focused. Another issue: the Bitter Lesson is primarily about compute scaling. Nick's paper is primarily about data quality. These aren't the same thing. More compute doesn't automatically mean better philosophical output — it depends on what the compute is doing. The Bitter Lesson's emphasis on search and learning might not map cleanly onto the way LLMs use the philosophical corpus. And a third issue: the Bitter Lesson could be seen as undercutting Nick's emphasis on the distinctive features of philosophy. If general methods work for everything (chess, Go, vision, language, science), then the fact that they work for philosophy too might not require a special explanation. Nick's elaborate argument about the virtue-filtered corpus might be unnecessary if the Bitter Lesson already predicts the outcome on general grounds. Why do we need to explain WHY general methods work for philosophy if they work for everything? But I think Nick could respond: precisely because philosophy is the hardest case. Chess, Go, and image recognition have objective success criteria. Philosophy's success criteria are contested and domain-internal. If the Bitter Lesson works even here, that tells us something about the nature of those criteria — that they are, despite appearances, encoded in the data. And that IS the argument of the paper. Let me step back and think about what's most interesting here... I think the richest connections are: 1. The paper provides a MECHANISM for the Bitter Lesson in philosophy's case (the virtue-filtered corpus) 2. The Bitter Lesson provides additional ammunition against Zahavy (you're making the same mistake as the chess researchers) 3. The Bitter Lesson predicts that prompting will become less important over time — which STRENGTHENS rather than weakens Nick's argument 4. Nick's framework could illuminate why the Bitter Lesson works generally — data-generating processes in mature domains pre-filter for quality 5. The Bitter Lesson applied to philosophy has striking metaphilosophical implications about the nature of philosophical knowledge 6. The phenomenological limit is the genuinely interesting test case — the Bitter Lesson predicts it will fall; Nick's analysis suggests it's more robust than other limits people have identified There's also a rhetorical possibility I shouldn't ignore: the Bitter Lesson is well-known in AI/tech circles but not in philosophy. Connecting the paper to it could increase the paper's reach and appeal to an interdisciplinary audience. Let me also think about whether this works better as part of the paper itself or as a separate Substack essay or ancillary argument... For the paper: it could work as a framing device in the introduction or as additional support in Section 4. The gluon scattering example already gestures at the Bitter Lesson without naming it. Making the connection explicit would ground the paper in a known pattern from AI research. For a Substack essay: this might be the better venue, actually. The paper is already dense. The Bitter Lesson connection is more of a contextualizing move than a philosophical argument. It might work better as a public-facing piece that introduces the paper's ideas to a broader audience. Or — and this might be the most interesting option — the connection could generate a genuinely new thread of argument that neither the paper nor the Bitter Lesson currently contains. Something about what happens when the Bitter Lesson is applied reflexively to the domain of philosophical method itself. If general methods beat domain-specific methods in AI, does that mean general philosophical methods (broad training, diverse exposure) beat domain-specific ones (narrow expertise, specialist knowledge) in philosophy too? This would be a contribution to the debate about philosophical methodology — connecting it to Williamson's argument against intuition-based methods and toward broader evidence bases. Actually, that last point is really interesting. Williamson argues against "armchair" methods that rely on intuitions and for methods that draw on broader evidence. The Bitter Lesson says general methods beat specialist ones. These are structurally similar claims. Is there a deep connection between Williamson's methodological pluralism and the Bitter Lesson? Could Nick argue that the Bitter Lesson provides empirical support for Williamson's methodological position? This is getting deep. Let me also consider whether there's a way to formulate the connection that would be surprising or non-obvious even to someone who knows both the paper and the Bitter Lesson... The most surprising version might be: the Bitter Lesson is not just an analogy for what's happening in LLM philosophy — it's an instance of the same phenomenon. The Bitter Lesson says that in AI, hand-coded expert knowledge loses to general learning from data. Nick's paper says that in philosophy, individual expert insight is less important than the accumulated wisdom of the filtered corpus. Both are claims about the superiority of distributed, evolved knowledge over individual, designed knowledge. The Bitter Lesson is, in a sense, a philosophical claim about epistemology — and Nick's paper is providing the philosophical analysis that the Bitter Lesson lacks. OK, I think I've explored this enough to give a structured answer. Let me also note what I'm uncertain about: whether Nick sees the connection primarily as (a) the Bitter Lesson supporting his paper, (b) his paper explaining the Bitter Lesson, (c) both generating a new insight that neither contains alone, or (d) something else entirely. ## The Generating Philosophy Paper and the Bitter Lesson I think there are genuinely striking connections here, running in multiple directions. Let me lay them out — I see at least six threads worth considering, and they range from "straightforward support" to "potentially paper-altering." ### 1. The virtue-filtered corpus as a mechanism for the Bitter Lesson Sutton observes that general methods leveraging computation beat domain-specific methods encoding human knowledge. He doesn't explain WHY. Your paper provides a mechanism — at least for philosophy. The philosophical corpus has been filtered by peer review, citation, teaching, and anthologizing for properties that track Williamson's intrinsic virtues. Statistical plausibility over that corpus converges with philosophical quality. So the reason general learning works for philosophy is that the domain's own quality-control has pre-shaped the data. The system doesn't need domain knowledge built in because the data already encodes it. This is more than analogy. Your paper could be read as explaining a specific instance of the Bitter Lesson — and in doing so, suggesting a general explanation that Sutton lacks: the Bitter Lesson works in mature domains because the data-generating processes in those domains have already filtered for quality. The statistical regularities the model learns aren't noise; they're the sedimented product of centuries of expert evaluation. ### 2. Ammunition against Zahavy The Bitter Lesson provides a structural response to Zahavy that's distinct from (and complementary to) your existing response. Zahavy argues that theoretical breakthroughs require "manipulative abduction" — embodied simulation. The Bitter Lesson identifies this as a recurring mistake: researchers think they know what process is required (strategic understanding for chess, intuition for Go, embodied simulation for physics) and they're wrong. General methods achieve the same outcomes through different processes. Einstein needed to imagine the elevator because he was a human with limited computational resources. That doesn't establish that embodied simulation is the only route to the equivalence principle — it establishes that it was Einstein's route. Your existing response works by arguing philosophy's inputs are propositional and available. The Bitter Lesson response works differently: it says Zahavy is making an unwarranted inference from "this is how humans did it" to "this is how it must be done." The history of AI shows that inference is unreliable. These two responses could layer — yours addresses the specific question of philosophy's inputs; the Bitter Lesson addresses the general form of the argument. ### 3. The prompting paradox — the Bitter Lesson strengthens your argument by threatening it Your Section 4 develops a prompting taxonomy: dialectical framing, solution-gestured prompting, conversational iteration. The Bitter Lesson predicts this taxonomy is temporary — that as models scale, skilled prompting will matter less. The default output will improve. The "generic, hedging text" you describe as the unprompted default is an artifact of current capability, not a permanent feature. This initially looks like a problem. But it actually strengthens the argument: the Bitter Lesson predicts that LLMs will get BETTER at philosophy over time, not worse. The convergence between statistical plausibility and philosophical quality will only deepen with more data and more compute. If you're right about the virtue-filtered corpus, then the Bitter Lesson predicts a trajectory — from needing skilled prompting now, to needing less and less as scale increases. The paper could present Section 4's prompting taxonomy as a description of current practice while acknowledging that the Bitter Lesson predicts it will become less necessary. This is a feature, not a bug. ### 4. The phenomenological limit as the genuinely interesting test case This is where the tension between your paper and the Bitter Lesson is productive rather than destructive. The Bitter Lesson would predict that even the phenomenological limit falls — that the Merleau-Ponty boundary you identify (philosophy that requires first-person attention to discover something not yet articulated) will be overcome by scale. Sutton's critics have said "surely AI can't do X" for every X, and been wrong. But your analysis suggests this limit is different from the chess/Go limits. It's not a limit of computational difficulty — it's a limit of data availability. No amount of compute generates an observation that hasn't been made and described. The Bitter Lesson says "more compute beats hand-coding." It doesn't say "more compute beats the absence of data." However — and this complicates things — multimodal models and embodied AI might eventually dissolve this limit through different means. Not by learning from philosophical text, but by acquiring something functionally equivalent to phenomenological attention through other modalities. Your paper's limit is about text-trained systems. The Bitter Lesson might apply to the broader trajectory of AI, including modalities your paper deliberately sets aside. This could be worth acknowledging — the limit is robust for LLMs as currently constituted, but the Bitter Lesson suggests it may not be permanent across all AI architectures. ### 5. The metaphilosophical implication — what philosophy IS This might be the deepest thread. If the Bitter Lesson applies to philosophy, what does that say about the nature of philosophical knowledge? It suggests that the philosophical tradition — the corpus, the filtering, the accumulated practice — encodes more than any individual philosopher explicitly knows. Individual genius is less important than the evolved structure of the discourse. The intrinsic virtues are properties of the tradition, not properties of individual minds. When Nick writes that "the virtues are latent in the model: implicit in the statistical regularities, recoverable from outputs, but not explicitly represented as rules" — this is the Bitter Lesson applied to philosophical knowledge itself. This has a Darwinian flavour: philosophical arguments compete for survival in a selection environment (peer review, citation, teaching), and the surviving corpus encodes the fitness criteria. An LLM trained on this corpus inherits the accumulated selection, just as a Bitter Lesson-style system inherits the structure of its domain through data rather than through explicit encoding. The tradition is smarter than any of its participants. This connects to — but is distinct from — Williamson's methodological position. Williamson argues for broader evidence bases and against narrow intuition-based methods. The Bitter Lesson provides empirical support for this: general methods (broad training) beat specialist methods (narrow expertise). Could Nick argue that the Bitter Lesson provides independent, empirical evidence for the kind of methodological pluralism Williamson advocates? ### 6. The general-distribution question and Sellars Your Section 4 asks whether a general-distribution LLM (trained on everything) does better than a philosophy-specialist model. The Bitter Lesson predicts: the general model wins. And your Sellars quotation explains why — philosophy is about "how things in the broadest possible sense hang together in the broadest possible sense." A system trained on the full breadth of human knowledge has been trained on philosophy's own subject matter. Specialist training might actually NARROW the distribution away from the cross-domain connections that philosophical argumentation draws on. This is a testable empirical prediction that the Bitter Lesson and your framework jointly generate: fine-tuning a model exclusively on philosophy should produce WORSE philosophical output than the base model, because it loses the breadth that Sellars identifies as philosophy's subject matter. If this prediction holds, it would be striking confirmation of both claims. --- ### Where could this go? I see three options for how this connection could function: The Bitter Lesson as a framing device in the introduction — replacing or supplementing the gluon scattering example. "The bitter lesson of AI research is that general methods leveraging computation beat domain-specific methods encoding human knowledge. We argue that this lesson applies to philosophy — and more strongly than to other domains, because philosophy's evaluative standards are already encoded in its textual corpus." This would ground the paper in a known pattern and signal interdisciplinary relevance. The Bitter Lesson as additional support in Section 4 — strengthening the prompting taxonomy by contextualizing it within the broader trajectory of AI capability, and bolstering the Sellars point about general vs. specialist training. As a separate Substack essay — the paper is already dense. A public-facing piece connecting the paper's ideas to the Bitter Lesson could introduce the arguments to a tech/AI audience that knows Sutton but not Williamson. This might be the most practical option if the paper's structure is already near-settled. Or there's a possibility I find the most interesting: the connection generates a genuinely new argument that neither the paper nor the Bitter Lesson currently contains — about what happens when you apply the Bitter Lesson reflexively to philosophical method itself. If general methods beat domain-specific methods, and if philosophy's own methodological debates are about how specific vs. how general one's evidence base should be, then the Bitter Lesson is itself a contribution to philosophy of philosophy. ### Remaining uncertainties I'm not sure whether you see the connection as primarily (a) the Bitter Lesson supporting the paper, (b) the paper explaining the Bitter Lesson, (c) both jointly generating something new, or (d) a Substack thread rather than a paper thread. The direction matters for how it gets developed. I'm also not sure how the Bitter Lesson interacts with the co-authorship with Enrico — does the Bitter Lesson framing fit the paper's audience (philosophy journals), or is it more suited to a tech-adjacent audience? *La lezione amara suggerisce che il sapere accumulato nelle nostre biblioteche filosofiche potrebbe contenere più saggezza di quanto i singoli filosofi che lo hanno prodotto abbiano mai inteso — una forma di conoscenza distribuita che nessun individuo possiede ma che il corpus, nel suo insieme, custodisce.* ### User take a look at today's daily note to see a chat i just had with another llm about this subject. the last couple of turns are particularly relevant i think ### Assistant OK, I've read the full exchange. The Raycast conversation gets to the right destination eventually — Sellars as the bridge, the virtue-filtered corpus as the mechanism — but it takes a lot of pushing and it stays fairly close to the surface once it arrives. Let me say what I think it missed, and where the really interesting possibilities are. The Raycast conversation settles on a reading where the three pieces relate like this: Sellars provides the definition, Sutton provides the prediction, the paper provides the mechanism. That's correct but it's only the first layer. There are several things it doesn't get to. First thing. The Raycast conversation treats the Bitter Lesson as external support for the paper — as if the paper is already complete and the Bitter Lesson is a nice bonus that confirms it. But I think the more interesting possibility is the reverse: the paper provides something the Bitter Lesson lacks. Sutton's essay is an empirical generalization. It observes a pattern — general methods win — but doesn't explain why. Why SHOULD scaling up produce better results than encoding expert knowledge? The Bitter Lesson just says: look at the track record. It's essentially an induction. The paper's virtue-filtered corpus argument is an explanation. It says: general methods work for philosophy because the domain's own quality-control processes have pre-shaped the data. The statistical regularities the model learns aren't arbitrary patterns; they're the downstream effects of centuries of expert evaluation that selected for loveliness, explanatory power, non-ad-hocness. The reason the "dumb" method works is that the data is smart. And this isn't just an explanation for philosophy. It's a candidate explanation for the Bitter Lesson in general. Why did general methods win in chess? Because the corpus of recorded chess games is a corpus filtered by competitive play — bad moves get punished, good moves get rewarded, and the surviving record encodes strategic quality. Why did general methods win in speech recognition? Because the corpus of recorded speech is a corpus filtered by communicative success — utterances that don't convey meaning don't get preserved and reproduced. In each case, the "general method" works because the data isn't raw — it's the output of a domain-internal selection process that has filtered for the properties that constitute quality in that domain. The Bitter Lesson is true because data-generating processes in mature domains do the expert's job before the model sees the data. If this is right, then Nick's paper isn't just an application of the Bitter Lesson to philosophy. It's potentially an explanation of why the Bitter Lesson holds. That's a much more interesting relationship. Second thing. The double meaning of "hanging together." The Raycast conversation notes that LLMs learn how things "hang together" in text. But there are two senses of "hanging together" in play, and the paper's argument is precisely about their convergence. Sense 1 (statistical): words, phrases, argumentative moves that tend to co-occur, that predict each other, that are statistically associated in the training distribution. This is what the model learns. Sense 2 (philosophical): genuine dependence relations between phenomena — how one thing bears on another, how things are connected by explanatory, causal, logical, or conceptual relationships. This is what Sellars says philosophy aims to understand, and what Dellsén et al. say philosophical progress consists in representing. The paper's argument is that these two senses converge in the philosophical corpus. Statistical hanging-together (Sense 1) tracks philosophical hanging-together (Sense 2) because the corpus has been filtered for texts where the argument actually works — where the connections are genuine, not merely rhetorical. In a corpus filtered for loveliness, the statistically probable continuation IS the philosophically illuminating one. This double meaning of "hanging together" is, I think, the real hinge of the Sellars-Bitter Lesson connection, and the Raycast conversation only gestures at it. Third thing. The "broadest possible sense" and the specialist-vs-generalist prediction. Section 4 of the paper already uses the Sellars quote to motivate the question of whether general-distribution LLMs outperform specialist ones. The Bitter Lesson provides a strong prediction here: the general model wins. And Sellars explains why: because philosophy IS the discipline of maximal breadth. A model trained on everything has been trained on philosophy's own subject matter — not as a side effect but constitutively. Philosophy is the discipline that needs the most context, the most cross-domain connections, the broadest base of materials. A specialist model, by narrowing its training distribution, would lose the very breadth that Sellars identifies as philosophy's aim. This generates a testable empirical prediction: fine-tuning a model exclusively on philosophy should produce WORSE philosophical output, not better, because it cuts the model off from the cross-domain connections that philosophical argumentation draws on. This is a non-obvious prediction that follows from the Sellars + Bitter Lesson combination. If it turned out to be true, it would be striking confirmation. Fourth thing. The Bitter Lesson as a response to Zahavy that's distinct from the paper's existing response. The paper responds to Zahavy by arguing that philosophy's starting points are propositional and linguistically available. That's a response about DATA — about what's in the corpus. The Bitter Lesson provides a structurally different response, about PROCESS. Zahavy says theoretical innovation requires embodied simulation — manipulative abduction. The Bitter Lesson says: you're making the same mistake as the chess researchers who said chess requires strategic understanding, or the Go researchers who said Go requires intuition. You're observing how HUMANS do it and concluding that's the only way it can be done. The history of AI shows that inference doesn't hold. General methods achieve the same outcomes through different processes. These two responses are complementary. The paper's existing response says: philosophy doesn't need what Zahavy claims it needs (embodied simulation). The Bitter Lesson response says: even if it DID need something like that, you can't conclude from the human case that no other route exists. Fifth thing. The prompting taxonomy and the trajectory. The Raycast conversation notes that the Bitter Lesson predicts prompting will become less important over time. But there's a more interesting way to frame this. Section 4 describes a spectrum of prompting modes — from bare questions (which produce generic output) through dialectical framing to conversational iteration. The Bitter Lesson predicts this spectrum will compress over time: what currently requires dialectical framing will eventually emerge from bare questions, because the model will get better at accessing the virtue-dense regions of the distribution without being guided there. This means the paper's Section 4, if the Bitter Lesson is right, is describing a TRANSITIONAL state — the current moment where human philosophical skill is still needed to activate the model's capacity. The trajectory is toward less scaffolding, not more. And the paper could acknowledge this: what it describes as current practice will evolve. The prompter's role will diminish. But here's the twist: will it diminish to zero? The Bitter Lesson's strongest form says yes — general methods will eventually handle everything. But Nick's phenomenological limit suggests a permanent role for humans: originating observations that require first-person attention. So the paper could engage the Bitter Lesson's strongest form and argue that it's almost right — but that there's an irreducible residue at the phenomenological boundary that even infinite scale can't eliminate (for text-based systems). Sixth thing. The reflexive dimension. The paper already notes that it's self-exemplifying — produced by human-LLM collaboration, submitted for blind review. The Bitter Lesson adds a temporal dimension to this self-exemplification. If the Bitter Lesson is right, then papers like Nick's represent an early stage of a trend. Future philosophical work will involve even more LLM contribution and less human scaffolding. The paper isn't just an argument FOR the possibility of LLM philosophy — it's an early instance of the Bitter Lesson playing out in philosophy in real time. The fact that it requires a skilled human philosopher as prompter is the transitional feature that the Bitter Lesson predicts will diminish. Seventh thing. Where should this go — in the paper, or elsewhere? The Sellars quote is already in Section 4. The Bitter Lesson could enter there too, as a frame for the two empirical questions about general vs. specialist models. But it could also go in the Introduction — the gluon scattering example already gestures at the broader pattern of AI success. Making the Bitter Lesson explicit would give the paper a stronger opening frame: "There is a pattern in AI research. General methods always win. We argue this pattern extends to philosophy, and we explain why." Or it might work better as a separate piece — a Substack essay that introduces the paper's ideas through the Bitter Lesson frame, aimed at an interdisciplinary audience that knows Sutton but not Williamson. The paper itself is already dense. Adding another thread might overload it. Or — and this is what I find most interesting — it might generate an entirely new argument that doesn't belong in this paper but could be a follow-up. Something about the Bitter Lesson as a contribution to metaphilosophy: what the success of general methods in philosophy tells us about the nature of philosophical knowledge. If the Bitter Lesson applies to philosophy, it suggests that philosophical knowledge is more distributed, more tradition-encoded, more data-like than the rationalist picture (individual minds grasping truths through reason) allows. It's a deflationary picture of philosophical genius but an inflationary picture of the philosophical tradition. The tradition is smarter than any of its participants. And that's a philosophical claim worth developing. Let me also think about whether there are tensions or problems with drawing this connection too tightly... One potential issue: the Bitter Lesson is maximally bold. It says general methods ALWAYS win. Nick's paper is more nuanced — it identifies genuine limits (the phenomenological boundary). Tying the paper too closely to the Bitter Lesson might make it seem more aggressive than it is, or might invite the objection "but the Bitter Lesson is just an induction, and your paper's careful argument about the virtue-filtered corpus is more rigorous than that." The paper might not want to lean on the Bitter Lesson's authority, because it can stand on its own arguments. Another potential issue: audience. Philosophers may not know or care about the Bitter Lesson. Invoking it in the paper itself might feel like importing a claim from a different discourse. It might work better in the Introduction as a scene-setting observation, or in a footnote, than as a load-bearing part of the argument. A third issue: the Bitter Lesson is usually read as a prescriptive claim about AI research methodology. Nick's paper isn't about AI methodology — it's about philosophical methodology and evaluation. Transposing the Bitter Lesson from "how to build AI" to "what AI can do" requires some care. But I think the deepest and most interesting possibility is that the paper enriches the Bitter Lesson rather than just being supported by it. The virtue-filtered corpus argument is a PHILOSOPHICAL explanation of an empirical pattern in AI research. If Nick can develop that, it's a genuine contribution to two literatures at once. Here's what I think the Raycast conversation gets right and where it stops short. ## What it gets right The final formulation is clean: Sellars provides the definition (philosophy = understanding how things hang together in the broadest sense), Sutton provides the prediction (general methods win), the paper provides the mechanism (the virtue-filtered corpus). That's the basic triangle and it works. ## What it misses ### 1. The paper explains the Bitter Lesson, not the other way round The Raycast conversation treats the Bitter Lesson as external validation of your argument. But the more interesting direction runs the other way. The Bitter Lesson is an empirical generalization without an explanation — Sutton just says "look at the track record." Your virtue-filtered corpus argument is a candidate explanation for WHY general methods work. They work because data-generating processes in mature domains have already filtered for quality. The statistical regularities aren't noise; they're sedimented expert judgement. This isn't specific to philosophy. Chess games are filtered by competitive play. Recorded speech is filtered by communicative success. In every case where the Bitter Lesson holds, you can tell a version of your story: the "dumb" method works because the data is smart. The data has done the expert's job before the model sees it. If that's right, the paper doesn't just apply the Bitter Lesson to philosophy. It provides the philosophical analysis the Bitter Lesson has always lacked. That's a much more ambitious and interesting relationship. ### 2. The double meaning of "hanging together" There are two senses in play, and the paper's argument is about their convergence: - Statistical hanging-together: what the model learns — co-occurrence patterns, sequential dependencies, what predicts what in text - Philosophical hanging-together: what Sellars means — genuine dependence relations between phenomena, how things actually bear on one another The virtue-filtered corpus is the mechanism of convergence. In a corpus filtered for loveliness (Lipton), for non-ad-hocness (Williamson), for genuine illumination of dependence relations (Dellsén et al.), statistical probability TRACKS philosophical quality. The statistically probable continuation IS the philosophically illuminating one — not accidentally, but because the filtering has aligned the two senses. This double meaning is the real hinge. The Raycast conversation gestures at it but never makes it explicit. ### 3. A distinct Zahavy response Your existing response to Zahavy is about data — philosophy's starting points are propositional, available in the corpus. The Bitter Lesson provides a response about process. Zahavy argues that theoretical innovation requires embodied simulation. The Bitter Lesson says: that's the same mistake the chess researchers made. They observed how humans do it and concluded that's the only way. General methods achieve the same outcomes through different routes. You can't infer from "Einstein needed the elevator" that embodied simulation is the only path to the equivalence principle. These two responses layer. Yours addresses what philosophy needs as INPUT. The Bitter Lesson addresses the inference pattern Zahavy relies on. ### 4. The specialist-vs-generalist prediction The Sellars quote is already in Section 4, motivating the question of whether general-distribution models outperform specialist ones. The Bitter Lesson + Sellars jointly generate a strong and testable prediction: fine-tuning on philosophy should produce WORSE philosophical output. Because philosophy IS the broadest discipline, a model that loses breadth loses what philosophy needs. A model trained on everything has been trained on philosophy's own subject matter — that's what Sellars' double "broadest possible sense" means. If this prediction turned out to be true, it would be striking confirmation of both claims simultaneously. ### 5. The trajectory of prompting The Bitter Lesson predicts Section 4's prompting taxonomy is transitional. What currently requires dialectical framing will eventually emerge from bare questions, because scaling up will give the model better access to virtue-dense regions of the distribution without human guidance. Your paper could frame Section 4 as describing the current state while acknowledging this trajectory. And the phenomenological limit becomes the interesting test: the Bitter Lesson says ALL human scaffolding becomes redundant; your paper says there's an irreducible residue at the phenomenological boundary. That tension is productive, not embarrassing. ## Where could this go? I see three options that aren't mutually exclusive: In the paper itself — probably in the Introduction, where the gluon scattering example already gestures at the pattern. Making the Bitter Lesson explicit would give the paper a stronger opening frame and connect it to a debate that's well-known outside philosophy. Or in Section 4, near the Sellars quote that's already there, where it would strengthen the argument about general vs. specialist models. As a Substack essay — introducing the paper's ideas to an interdisciplinary audience through the Bitter Lesson frame. This might be the most natural home for the broader "the paper explains the Bitter Lesson" claim, which is too sweeping for the paper itself. As a new thread — the Bitter Lesson as a contribution to metaphilosophy. If general methods work for philosophy, what does that say about what philosophical knowledge IS? It suggests the tradition is smarter than any of its participants. That the corpus encodes more than any individual philosopher intended. That philosophical knowledge is distributed, evolved, data-like — more like what emerges from a selection process than what emerges from individual rational insight. That's a philosophical claim worth developing separately. The possibility I find most interesting is the second direction — the paper explaining the Bitter Lesson rather than merely being supported by it. That reversal is where the genuinely novel contribution would be. *Se il sapere filosofico è davvero sedimentato nel corpus come le venature nel marmo, allora il metodo più 'stupido' — la predizione statistica su larga scala — potrebbe estrarre ciò che generazioni di singoli filosofi hanno faticato a vedere.* ### User I feel I need to learn more about the bitter lesson and how it has affected thinking about ai since it was released (today's date is 30 Mar 2026). i am oging to ask a web based llm to conduct some deep research for me. Can you help me write a good prompt that will find me information which will help my general knowledge but will also be created with my particular interest in the essay in mind. make sure that you explain your analysis, justification, reasons etc etc. for your answer BEFORE giving me your final answer. Please give me a long, deep, and detailed answer, with lots of options/ideas: as we are in the 'kicking ideas around' stage, I want as much feedback as you can give me so that i have a lot to consider. ### Assistant Let me think about what makes a good deep research prompt for this specific situation. There are several things in play: (1) Nick needs general knowledge about the Bitter Lesson and its reception, (2) he has a specific interest — the generating philosophy paper — that should shape what the research surfaces, and (3) the prompt needs to be structured so that a web-based research tool (probably Gemini Deep Research) produces reliable, well-sourced results rather than hallucinating. Let me start with what Nick actually needs to know, given the conversation we've just had and the paper's argument. There are general knowledge gaps: - The original essay itself (he got a decent summary from Raycast) - How has the Bitter Lesson been received since 2019? Who endorsed it, who pushed back? - Has the concept evolved? Sutton wrote it in 2019 — has the explosion of LLMs changed how people interpret it? - What are the strongest criticisms? The "architecture is also human knowledge" objection (transformers, attention mechanisms). The data efficiency objection. The environmental cost objection. - Has anyone formalized it? Is it just a blog post, or has it entered academic discourse? But then there are the angles specific to Nick's interest — things the prompt should steer the research toward without being so narrow that it misses the general context: - Has anyone applied the Bitter Lesson to non-STEM domains? To creative work, language, reasoning, argumentation? This matters because Nick would be extending it to philosophy. - Has anyone discussed the MECHANISM by which general methods win? This is the virtue-filtered corpus idea — that the data is pre-shaped by domain-internal quality-control. If anyone has made this argument, Nick needs to know, both to cite and to differentiate. - Has anyone connected the Bitter Lesson to questions about the nature of knowledge or expertise? If the Bitter Lesson is right, what does that say about what domain knowledge IS? This is the metaphilosophical angle. - The scaling hypothesis / scaling laws — how do these relate to the Bitter Lesson? Kaplan et al., Chinchilla, etc. The Bitter Lesson was pre-GPT-3. The scaling laws gave it empirical teeth. - Has anyone discussed the relationship between data quality and the Bitter Lesson? The standard reading is "more compute wins." But there's a subtler point about what the compute is operating ON. The "data is smart" angle. - RLHF and constitutional AI — these are interesting because they're a form of filtering. They shape the distribution the model operates over. Is this a form of "hand-encoding" (which the Bitter Lesson says is futile) or is it more like the virtue-filtering that Nick describes? - Has anyone in philosophy of AI or philosophy of mind engaged with the Bitter Lesson specifically? Now, let me think about what makes a good prompt for a deep research tool... The main failure modes of deep research tools are: 1. Hallucinating sources — they invent papers, misattribute claims, fabricate citations 2. Going too broad/shallow — giving a Wikipedia-level summary instead of finding specific arguments and responses 3. Missing recent material — the Bitter Lesson has been heavily discussed since GPT-3 (2020), GPT-4 (2023), and the subsequent scaling debates. A lot of the interesting reception is in blog posts, talks, and industry commentary, not just academic papers. 4. Conflating the Bitter Lesson with general "AI scaling" discourse — the specific essay has specific claims that are distinct from the broader scaling conversation 5. Anchoring on the most obvious/popular takes and missing the more interesting ones To guard against these: - Ask explicitly for sources with URLs, publication dates, and author names - Ask for direct quotations where possible (harder to hallucinate) - Distinguish between the essay itself, academic responses, and industry/blog discourse - Ask for BOTH supporters and critics — don't let it give a one-sided picture - Ask for specific claims and arguments, not just summaries - Be explicit about the time range: March 2019 to March 2026 Now, how to structure the prompt to serve both purposes — general knowledge AND the specific interest in philosophy... Option A: One big prompt that mixes both. Risk: the tool might lose focus, try to do everything, and do nothing well. Option B: Two separate prompts — one for general reception, one for the philosophy/knowledge angle. Risk: duplicated work, and the two reports might not connect well. Option C: A layered prompt — start with the general question, then add specific sub-questions that steer toward the interesting angles. This is probably best. The general question gives the tool its bearings; the sub-questions ensure it doesn't miss what matters. Let me also think about what Nick already knows and doesn't need repeated back to him. He knows the basic argument of the essay. He's read the Raycast summary. He doesn't need "what is the Bitter Lesson" explained again. What he needs is: reception, criticism, evolution, and — specifically — whether anyone has connected it to the nature of knowledge/expertise in non-engineering domains. There's also a question about what kind of sources will be most useful. Academic papers engaging with the Bitter Lesson would be ideal for citation purposes. But a lot of the interesting discourse is in: - Blog posts by AI researchers (Sutton himself, but also Gwern, Nostalgebraist, various alignment researchers) - Talks and lectures (Sutton has given talks elaborating on the essay) - Industry commentary (OpenAI's scaling philosophy, DeepMind's approach) - Philosophy of AI papers that may not cite the Bitter Lesson by name but engage with the same ideas The prompt should ask for all of these. Let me also think about potential pitfalls specific to this topic: - The Bitter Lesson is sometimes confused with the "scaling hypothesis" (which is related but distinct). The Bitter Lesson is about general methods vs. hand-coding; the scaling hypothesis is about model performance improving predictably with scale. They're related but the Bitter Lesson is the older, broader claim. - Some people read the Bitter Lesson as "just scale compute" when Sutton actually emphasizes SEARCH and LEARNING as the two methods that scale. The distinction between search (like MCTS in AlphaGo) and learning (like gradient descent) matters. - The essay has been cited in very different contexts — by people arguing for scaling up LLMs, by people arguing against AI safety research, by people arguing for or against specific architectural choices. Nick needs to understand these different appropriations. Now, there's a particular angle I think the prompt should push on that connects directly to the paper: has anyone argued that the Bitter Lesson works BECAUSE of data quality rather than just data quantity? This is the crux. The standard reading of the Bitter Lesson is quantitative — more compute, more data, better results. Nick's argument is qualitative — the data is pre-filtered for quality by domain-internal selection processes. If anyone has made a version of this argument, that's directly relevant. Related: the concept of "data curation" in AI research. There's been a lot of recent work on data quality mattering as much as or more than data quantity (the Chinchilla paper, the Phi models from Microsoft which used "textbook quality" data, etc.). This is adjacent to Nick's argument — the idea that what's IN the data matters more than how much there is. Actually, this is really important. The Phi models (Phi-1, Phi-1.5, Phi-2, Phi-3) from Microsoft Research explicitly argued that training on "textbook quality" data could achieve results comparable to much larger models trained on more data. This is DIRECTLY relevant to Nick's virtue-filtered corpus argument — it's empirical evidence that data filtered for quality produces better results than raw scale. The prompt should surface this. Similarly, the debate around "synthetic data" — training models on data generated by other models — relates to the question of filtering. If you use a powerful model to generate training data for a weaker model, you're filtering for quality. This has been a big topic in 2024-2026. And there's the question of RLHF/RLAIF (reinforcement learning from human/AI feedback). This is another form of filtering — you're shaping the model's distribution by rewarding outputs that humans judge to be good. Is this "hand-encoding" (which the Bitter Lesson opposes) or is it "filtering the distribution for quality" (which is compatible with the Bitter Lesson)? Different people read this differently, and Nick should know about the debate. Let me also think about whether the prompt should mention Nick's paper at all. On one hand, mentioning it might bias the research tool toward confirming Nick's thesis rather than finding genuine counter-arguments. On the other hand, if the research tool doesn't know what Nick is looking for, it might not find the most relevant material. I think the best approach is to mention the philosophical interest WITHOUT mentioning the specific thesis. Something like: "I am particularly interested in whether the Bitter Lesson has implications for the nature of knowledge and expertise, and whether anyone has applied it to non-STEM domains, including philosophy and the humanities." This steers the research without biasing it. One more thing. The prompt should be explicit about what constitutes a good source. Deep research tools are more reliable when you tell them what you want: - Provide URLs for all sources - Distinguish between claims that are well-supported by evidence and claims that are speculative - Include the actual publication date for each source - When quoting, quote directly rather than paraphrasing Actually, wait — I should think about which deep research tool Nick is likely to use. He has the gemini-deep-research skill, which suggests Gemini. Gemini Deep Research has specific strengths and weaknesses: - Good at finding and synthesizing web sources - Tends to over-summarize and under-quote - Can produce reports that look authoritative but have fabricated citations - Works best when given specific questions rather than broad topics - Benefits from being told to organize findings by theme rather than chronologically So the prompt should have clear sub-questions, ask for specific attribution, and request thematic organization. Let me also think about the structure of the prompt itself. A good deep research prompt for something like Gemini Deep Research should have: 1. A clear top-level question 2. Context about why you're asking (enough to guide relevance, not enough to bias) 3. Specific sub-questions that ensure coverage of the angles you care about 4. Instructions about sourcing and evidence quality 5. Instructions about what NOT to do (don't just summarize the essay, don't hallucinate sources) Let me draft some candidate sub-questions: General knowledge: - What are the strongest published criticisms of the Bitter Lesson? - How has the essay's reception changed since GPT-3 / GPT-4 / later models? - Has Sutton himself revised, extended, or defended the thesis since 2019? - What is the relationship between the Bitter Lesson and the scaling laws literature? - How do proponents and critics of the Bitter Lesson differ on the role of architecture (e.g., the Transformer)? Specific to Nick's interest: - Has anyone argued that the Bitter Lesson works because of data QUALITY (not just quantity)? - Has anyone applied the Bitter Lesson to non-STEM domains — reasoning, creativity, language, argumentation, the humanities? - Has anyone discussed what the Bitter Lesson implies about the nature of human expertise or domain knowledge? - What is the relationship between the Bitter Lesson and data curation / "textbook quality" data (e.g., Microsoft's Phi models)? - Has anyone in philosophy of AI or philosophy of mind engaged with the Bitter Lesson? - How does RLHF/RLAIF relate to the Bitter Lesson — is it a form of "hand-encoding" or something else? The prompt should also ask about the relationship between the Bitter Lesson and other influential AI essays/concepts: - Scaling laws (Kaplan et al. 2020, Hoffmann et al. 2022) - "The Scaling Hypothesis" (Gwern) - Chinchilla and compute-optimal training - Emergent abilities in large language models - The debate around "data-centric AI" One thing I want to be careful about: the prompt shouldn't be so long that the research tool gets overwhelmed. There's a sweet spot between too vague (and you get Wikipedia-level results) and too detailed (and the tool fixates on one sub-question and ignores others). I'd aim for a prompt that's 300-500 words, with a clear main question and 8-12 specific sub-questions organized into themes. Let me also consider: should the prompt ask for a bibliography or reading list? That would be useful — a curated set of the most important responses, criticisms, and extensions, with URLs and brief descriptions. Nick could then follow up on the most relevant ones himself. Actually, yes. The prompt should explicitly ask for a bibliography. Deep research tools are better at surfacing sources than at analyzing them. Nick can do the analysis himself; what he needs is the map of the landscape. Let me think about one more angle. The Bitter Lesson has been used as an argument in the AI safety debate — some people argue that if general methods always win, then safety research that tries to constrain models in specific ways is doomed. Others argue the opposite — that the Bitter Lesson means we should focus on general alignment approaches rather than specific safeguards. This debate is probably not directly relevant to Nick's paper, but it's part of the landscape he should know about, and it might surface interesting claims about the nature of knowledge and constraint. OK, I think I have enough to structure a prompt. Let me also consider whether to write one prompt or suggest multiple options... Given that Nick said "kicking ideas around," I should offer a few options: - Option 1: A comprehensive single prompt that covers both general and specific angles - Option 2: A focused prompt just on the specific angles that matter for the paper - Option 3: Two separate prompts — one for general background, one for the philosophy/knowledge angle I'll present the comprehensive option as my recommendation but show why the alternatives might also be worth considering. One final thought: the prompt should guard against the deep research tool's tendency to treat the Bitter Lesson as settled truth. Nick needs the DEBATE, not just the thesis. The prompt should explicitly ask for strong counter-arguments and unresolved tensions. ## Designing the prompt The challenge is getting a deep research tool to do two things at once: give you a solid map of the Bitter Lesson discourse (2019–2026) AND surface material that's specifically relevant to the virtue-filtered-corpus argument and the Sellars connection. These pull in different directions — the first wants breadth, the second wants targeted depth. The prompt needs to serve both without collapsing into either. ### What the prompt needs to do Surface general knowledge you're missing: - Who pushed back, and on what grounds? - How has the thesis evolved with GPT-3, GPT-4, and later? - Has Sutton himself revised or extended the claim? - The relationship to scaling laws, Chinchilla, and compute-optimal training - The "architecture is also human knowledge" objection (transformers as a counter-example?) Surface material specific to your interest: - Has anyone argued the Bitter Lesson works because of data QUALITY, not just quantity? (This is directly your virtue-filtered corpus argument. If someone has made this point, you need to know — both to cite and to differentiate.) - The Phi models from Microsoft, which explicitly trained on "textbook quality" data and achieved disproportionate results. This is empirical evidence for your filtering claim. - Whether anyone has applied the Bitter Lesson to non-STEM domains: reasoning, language, creativity, the humanities, philosophy. - Whether anyone has discussed what the Bitter Lesson implies about the NATURE of expertise — not just "AI can replace experts" but "what does it tell us about what expert knowledge IS?" - How RLHF/RLAIF relates — is it "hand-encoding" (which the Bitter Lesson opposes) or is it distribution-shaping (which is compatible)? - Whether anyone in philosophy of AI or philosophy of mind has engaged with the essay. ### What the prompt needs to guard against Deep research tools have specific failure modes: They hallucinate sources. The prompt should ask for URLs and publication dates, and should tell the tool to flag uncertainty about sources rather than fabricating details. They over-summarize. You don't need another summary of the essay. The prompt should explicitly say: "I already know the basic argument. I need the RECEPTION, CRITICISM, and EVOLUTION." They flatten debate into consensus. The prompt should ask for the strongest counter-arguments alongside the endorsements. They anchor on the most obvious takes. The prompt should push for less-obvious connections — the data quality angle, the humanities applications, the epistemological implications. ### Structure A layered approach works best: a clear top-level question, context about your interest (enough to guide relevance, not enough to bias the results toward confirming your thesis), specific sub-questions organized by theme, and instructions about evidence quality. I'd aim for 400-600 words — long enough to be specific, short enough that the tool doesn't lose the thread. One important choice: whether to mention your paper. I'd recommend against mentioning the specific thesis (virtue-filtered corpus, convergence of statistical plausibility with philosophical quality). If you do, the tool will try to confirm it rather than finding genuine counter-evidence. But mentioning that you're interested in the philosophical and epistemological implications is fine — it steers without biasing. ### The prompt --- I need a comprehensive research report on Rich Sutton's "The Bitter Lesson" (published March 13, 2019) and its reception, criticism, and influence in the seven years since publication. I already know the basic argument (general methods leveraging computation beat hand-encoded domain expertise). What I need is the discourse AROUND the essay — who engaged with it, how the ideas evolved, where the debates are. Organize your findings around these themes: ## 1. Reception and criticism (2019–2026) - What are the strongest published criticisms of the Bitter Lesson? I want specific arguments, not just "some people disagree." - The "architecture is human knowledge" objection: the Transformer architecture (Vaswani et al. 2017) is itself a design choice encoding human insight. How have people reconciled this with the Bitter Lesson? - Has Sutton himself revised, extended, or qualified the thesis since 2019? - Which prominent AI researchers have endorsed or challenged the essay, and on what specific grounds? ## 2. Relationship to scaling laws and data-centric AI - How does the Bitter Lesson relate to the scaling laws literature (Kaplan et al. 2020, Hoffmann et al. 2022 / Chinchilla)? - Has anyone argued that the Bitter Lesson works because of data QUALITY rather than just data quantity? I'm interested in the idea that the training data is already pre-filtered for quality by domain-internal processes (peer review in science, competitive play in games, communicative success in language). Has anyone made this argument? - How do the "textbook quality" data results (Microsoft's Phi model series) and the broader data curation movement relate to the Bitter Lesson? Do they support it, complicate it, or refine it? - How does RLHF (reinforcement learning from human feedback) and similar alignment techniques relate? Is this a form of "hand-encoding" that the Bitter Lesson would predict is futile, or is it something different? ## 3. Applications beyond engineering - Has anyone applied the Bitter Lesson to non-STEM domains — reasoning, creativity, language, argumentation, the humanities, philosophy? - Has anyone discussed what the Bitter Lesson implies about the NATURE of human expertise and domain knowledge — not just "AI can replace experts" but "what does the success of general methods tell us about what knowledge IS"? - Has anyone in philosophy of AI, philosophy of mind, or epistemology engaged with the essay specifically? ## 4. The broader intellectual landscape - How does the Bitter Lesson relate to other influential ideas: "The Scaling Hypothesis" (Gwern), emergent abilities in LLMs, the "data-centric AI" movement? - Has the essay been used in the AI safety/alignment debate, and if so, how? ## Source requirements - Provide URLs and publication dates for all sources. - Distinguish between academic papers, blog posts/essays, talks, and industry commentary. - When you can, quote directly rather than paraphrasing. - If you're uncertain about a source or attribution, flag it explicitly rather than presenting it as confirmed. - Include a bibliography at the end organized by type (academic, blog/essay, talk, other). Do NOT just summarize the original essay. I need the landscape of response and engagement. --- ### Notes on this prompt A few things I built in deliberately: The data quality question (theme 2) is the one most directly relevant to your paper. The virtue-filtered corpus argument IS a claim about why the Bitter Lesson works — the data has been pre-shaped by domain-internal quality control. If anyone has made this argument (or something adjacent), the research should surface it. But I framed it as a question ("has anyone argued...") rather than stating your thesis, so the tool will search for it rather than just confirming it. The "nature of expertise" question (theme 3) is the metaphilosophical angle — what the Bitter Lesson implies about what knowledge IS. This is the Sellars connection point. Again, framed as a search rather than a claim. The Phi models and RLHF questions are targeted at finding empirical evidence you could use. Phi is evidence that data quality matters (supporting the filtered-corpus argument). RLHF is interesting because it's a form of post-training distribution-shaping — is that "hand-encoding" or "filtering"? The answer to that question bears on whether your argument is compatible with or in tension with the Bitter Lesson. The source requirements section is there to reduce hallucination. Deep research tools are significantly more reliable when you tell them to provide URLs and to flag uncertainty. ### Alternative approaches If the tool gives you too much general background and not enough on the specific angles, you could run a second, more targeted prompt: "Has anyone argued that the Bitter Lesson succeeds because training data in mature domains has been pre-filtered for quality by domain-internal selection processes? I'm thinking of the analogy: chess game records are filtered by competitive play, scientific papers by peer review, philosophical texts by citation and anthologizing. The 'general method' works not because it's indifferent to domain knowledge but because the data already encodes that knowledge. Find any arguments in this vicinity — they might not reference the Bitter Lesson by name." That second prompt is riskier (the tool might just generate a response agreeing with you rather than finding actual sources) but it would surface adjacent arguments if they exist. *Il buon ricercatore sa che la domanda giusta contiene già la metà della risposta — ma l'altra metà la si trova solo guardando dove nessuno ha ancora pensato di cercare.*