# IBE — The Full Architecture (Lipton) *Visual anatomy of Inference to the Best Explanation: the inference engine, the loveliness/likeliness distinction, and Bayesian integration — drawn from Chapters 4 and 7 of* Inference to the Best Explanation. > "According to Inference to the Best Explanation, our inferential practices are governed by explanatory considerations. Given our data and our background beliefs, we infer what would, if true, provide the best of the competing explanations we can generate of those data." — Lipton, Ch. 4 Source: [[Lipton - Ch04 Inference to the Best Explanation]] · [[Lipton - Ch07 Bayesian Abduction]] · [[Inference to the Best Explanation by Peter Lipton 2004]] --- ## I. The Two-Filter Inference Engine Lipton's core model: from raw evidence to epistemically warranted inference, governed throughout by explanatory considerations. Two filters operate sequentially — plausibility, then loveliness. The inference engine is not inference-then-explanation, but inference *via* explanation. ```mermaid flowchart TB E["🔍 EVIDENCE\nobservable phenomena"] subgraph GEN["① POOL GENERATION — Context of Discovery"] ALL["All logically compatible\npotential explanations\n(vast pool, includes 'crazy' options)"] PFILT["Plausibility Filter\nepistemic pre-selection\n(live options only)"] LIVE["Live Options Pool\nserious candidates"] ALL --> PFILT --> LIVE end subgraph EVAL["② LOVELINESS ASSESSMENT — Explanatory Virtues"] UNI["🔗 Unification\nscope: how many phenomena\ndoes H explain?"] SIM["✂️ Simplicity\nontological parsimony;\nfewer unexplained posits"] MECH["⚙️ Mechanism\nphysical/causal pathway\nspecified and credible?"] CONTR["⚖️ Contrastive Fit\nwhy P rather than Q?\ncontrast class explained?"] FERT["🌱 Fertility\npromise of further\nunpredicted explanations"] end subgraph SEL["③ COMPETITIVE SELECTION — Filter 2"] RANK["Rank competing explanations\nby overall loveliness"] BEST["Best Potential Explanation\nidentified"] TEST{"Lovely enough\nfor inference?"} RANK --> BEST --> TEST end subgraph OUTP["④ EPISTEMIC OUTCOME"] INF["✅ INFER\nbest potential explanation\nis actual explanation"] AGN["⏸ AGNOSTICISM\nno candidate\ngood enough; suspend judgment"] CONS["📐 Consequence Condition\ndeductive + explanatory\ndownstream inferences permitted\n(e.g. Newton → orbital laws)"] INF --> CONS end E --> ALL LIVE --> UNI & SIM & MECH & CONTR & FERT UNI & SIM & MECH & CONTR & FERT --> RANK TEST -->|"yes"| INF TEST -->|"no"| AGN SELF["♻ Self-Evidencing Loop\nTracks in snow / red-shift of galaxy:\nevidence partly constitutes support\nfor the explanation of that evidence"] CONS -.->|"enables"| SELF SELF -.->|"evidence ↔ explanation\nmutually supporting"| E WARN["⚠ Not: Inference to Best ACTUAL Explanation\n(circular & epistemically ineffective —\nrequires knowing truth before inferring it.\nPotential / actual distinction is essential.)"] PFILT -.->|"avoids this trap"| WARN classDef ev fill:#1e3a5f,stroke:#5b9bd5,color:#fff,stroke-width:3px classDef filt fill:#1a3a2a,stroke:#52b788,color:#fff,stroke-width:2px classDef virt fill:#2d1b69,stroke:#9d4edd,color:#fff classDef good fill:#1a4a2a,stroke:#52b788,color:#fff classDef loop fill:#2a2a1a,stroke:#d4a017,color:#ddd classDef warn fill:#3a1a1a,stroke:#e94560,color:#aaa,stroke-dasharray:5 5 class E ev class LIVE,BEST filt class UNI,SIM,MECH,CONTR,FERT virt class INF,CONS good class SELF,AGN loop class WARN warn ``` --- ## II. Loveliness vs. Likeliness — The Signal Distinction Chapter 4's pivotal move: "best" is ambiguous between *likeliest* (most probable given evidence) and *loveliest* (most illuminating if true). These come apart in instructive ways. IBE's interesting version is **Inference to the Loveliest Potential Explanation** — loveliness as a guide to likeliness, not a substitute for it. ```mermaid quadrantChart title Loveliness vs. Likeliness — Cases from Lipton x-axis Low Likeliness --> High Likeliness y-axis Low Loveliness --> High Loveliness quadrant-1 The goal: lovely and probable quadrant-2 Lovely but improbable quadrant-3 Low on both dimensions quadrant-4 Probable but uninformative Special Relativity: [0.92, 0.91] Newtonian Mechanics (classical): [0.87, 0.89] Newtonian Mechanics (post-Einstein): [0.34, 0.88] General Relativity: [0.90, 0.94] Conspiracy Theories: [0.09, 0.71] Dormitive Virtues (opium): [0.94, 0.05] Cartesian Demon Hypothesis: [0.05, 0.27] Ptolemaic Epicycles: [0.21, 0.31] Scientific Realism (IBE applied): [0.65, 0.78] ``` | | **Loveliness** | **Likeliness** | |---|---|---| | **Definition** | How explanatory H would be *if* true | Posterior probability given evidence | | **Relativised to** | The specific phenomenon E + evidence | Total available evidence | | **Affected by new competitors** | Rarely changes | Often decreases | | **Affected by new evidence** | Only indirectly | Directly via Bayes | | **Affected by designing for data** | Yes (accommodation vs prediction) | No (logical relation unchanged) | | **IBE's thesis** | Loveliness *guides* likeliness | ← Loveliness is a symptom of this | > **Dormitive virtues**: Opium's sleep-inducing power is maximally likely (trivially true) yet maximally unlovely — the very model of a bad explanation. > > **Conspiracy theories**: Unify many apparent coincidences under a single source — lovely, if true — yet typically very unlikely. > > **Newtonian mechanics post-Einstein**: Remains as lovely an explanation of classical data as ever; its likeliness diminished with new evidence and competitors. --- ## III. IBE as Bayesian Realization — Chapter 7 Lipton's irenic synthesis: IBE does *not* violate Bayesian rationality. The relationship is **realization** — explanatory considerations are the cognitive mechanism by which we enact Bayesian conditionalization. IBE is to Bayes what squash technique is to physics: the ball still obeys mechanics, but technique determines where it goes. ```mermaid flowchart TD subgraph PSYCH["Cognitive Psychology Constraint — Kahneman and Tversky"] LINDA["Linda Conjunction Fallacy\n85% rate conjunction more probable\nthan conjunct alone — violates\nbasic probability axiom"] BASE["Base Rate Neglect\nDoctors give 95% when answer\nis under 2% — ignoring p(H)"] REGRESS["Regression to Mean Ignored\nFlight instructors infer punishment\nworks — actually just regression\n(spurious causal explanation)"] RESULT["→ Humans are systematically poor\nat abstract probabilistic calculation,\nbut naturally and fluently\ndeploy explanatory heuristics"] LINDA & BASE & REGRESS --> RESULT end subgraph BAYES["Bayesian Conditionalization — Formal Constraint"] PH["p(H)\nPrior probability of H\n← shaped by simplicity,\nunification, past conditionalizing"] PE["p(E)\nPrior probability of E\n= surprisingness of data\nbefore E is observed"] PEH["p(E|H)\nLikelihood\nhow probable would E be\nif H were true?"] POST["p(H|E)\nPosterior Probability\n= Likeliness target\nthe Bayesian output"] PH & PE & PEH --> POST end subgraph IBEH["IBE — The Explanationist Heuristic"] POOLGEN["Pool Generation\nask: what would explain E?\n(context of discovery —\nBayes is silent here)"] LOVEVAL["Loveliness Assessment\nif H, how illuminating?\nhow well would it explain?"] SURPRE["Surprisingness of E\nexplanatory gap:\nhow much demands explanation?"] BESTID["Competitive Ranking\nbest potential explanation\nselected"] end LOVEVAL -->|"Simplicity + Unification\n+ Fertility shape priors"| PH LOVEVAL -->|"Mechanism articulation\n+ Contrastive fit shape likelihoods"| PEH SURPRE -->|"explanatory gap =\nlow prior p(E)"| PE POOLGEN -->|"determines which H\nare considered at all"| PH POST -->|"loveliness as barometer\nof posterior probability"| BESTID RESULT -->|"IBE fills the gap:\nexplanatory thinking is our\ncognitive route to Bayesian output"| LOVEVAL NOTE["Lipton's 4th Response (favoured):\nIBE doesn't violate Bayes — it realizes it.\nExplanatory considerations help inquirers\ndetermine: (1) priors, (2) likelihoods,\n(3) which evidence is relevant,\n(4) which hypotheses to generate.\nBayes constrains; IBE implements."] RESULT -.->|"why we need heuristics"| NOTE POST -.->|"what the heuristic approximates"| NOTE classDef psych fill:#4a1942,stroke:#d4a017,color:#fff,stroke-width:2px classDef bayes fill:#1a3a2a,stroke:#52b788,color:#fff,stroke-width:2px classDef ibe fill:#1e3a5f,stroke:#5b9bd5,color:#fff,stroke-width:2px classDef note fill:#2d2d1a,stroke:#d4a017,color:#ddd,stroke-dasharray:5 5 class LINDA,BASE,REGRESS,RESULT psych class PH,PE,PEH,POST bayes class POOLGEN,LOVEVAL,SURPRE,BESTID ibe class NOTE note ``` ### The Three Roles of Loveliness in Bayesian Conditionalization | Bayesian Component | What IBE Contributes | |---|---| | **p(H) — Prior** | Simplicity, unification, fertility determine prior plausibility; today's priors are yesterday's posteriors shaped by past explanatory success | | **p(E\|H) — Likelihood** | Mechanism articulation + contrastive fit: we judge "how likely would E be if H?" by asking "how well would H explain E?" | | **p(E) — Surprisingness** | The explanatory gap: data that cry out for explanation have low priors; our sense of surprisingness tracks this | | **Evidence selection** | We notice that a datum is relevant to H precisely by seeing that H would explain it (Sherlock Holmes: the dog that didn't bark) | | **Context of discovery** | Bayes says nothing about where H comes from; IBE generates candidates by asking "what would explain E?" | --- ## IV. The Objection Landscape ```mermaid mindmap root((IBE)) Attractions Self-evidencing explanations Tracks in snow Red-shift of galaxy Phenomenon supports its own explanation Explanatory detour Semmelweis and childbed fever We seek causes even when predicting Source of scientific predictive power Advances over competitors Blocks the raven paradox Covers vertical inference to unobservables Avoids HD model overpermissiveness Allows agnosticism unlike HD Self-applying Discovery of IBE is itself an IBE Applies to scientific realism argument Applies to external world realism Objections Triviality Just inference to likeliest cause in fancy dress? Must reveal symptoms of likeliness Must go beyond HD model Hungerford objection Beauty is in the eye of the beholder Loveliness too subjective Too interest-relative for objective warrant Voltaire objection Why should loveliest equal likeliest? Why assume we inhabit loveliest world? Why assume our candidates include the truth? van Fraassen challenge Bayesianism is normatively correct via dutch book IBE gives illicit posterior bonus Therefore IBE is pragmatically irrational Responses to van Fraassen R1 Bayesianism fails on its own terms Old evidence problem New theory problem Conjunction inflation R2 Descriptive psychology favours IBE Kahneman and Tversky Humans use explanatory not Bayesian heuristics R3 Bayes underdetermines inference Consistent with explanationist supplement Priors not anchored by probability theory R4 IBE realizes Bayesianism Liptons favoured response Explanatory heuristic runs conditionalization Compatible not competing ``` --- ## V. Contrastive Inference — The Key Specimen Chapter 5 uses **Semmelweis and childbed fever** as the paradigm case of IBE in action. The structure of contrastive inference maps perfectly onto IBE's machinery. ```mermaid flowchart LR subgraph FACT["THE FACT TO EXPLAIN"] F1["Why did women in Ward 1\ncontract childbed fever\nat much higher rates\nthan women in Ward 2?"] end subgraph CONTRAST["CONTRASTIVE STRUCTURE"] P["Fact: P\nWard 1: high fever rate"] Q["Foil: Q\nWard 2: low fever rate"] PQ["Explain P-rather-than-Q\nnot just why P"] P & Q --> PQ end subgraph CANDIDATES["COMPETING POTENTIAL EXPLANATIONS"] H1["Epidemic atmospheric\ninfluences (endemic)"] H2["Overcrowding / poor\nventilation"] H3["Psychological distress\nfrom priest's bell"] H4["Delivery position\nin Ward 1 vs Ward 2"] H5["Cadaveric infection\nfrom doctors' hands\n(Semmelweis's hypothesis)"] end subgraph TESTS["CONTRASTIVE TESTS — IBE in Action"] T1["Epidemic: affects\nboth wards equally\n→ fails contrast ✗"] T2["Ventilation: both\nwards same building\n→ fails contrast ✗"] T3["Bell: remove bell\n→ fever rate unchanged ✗"] T4["Position: change\nposition in Ward 1\n→ no change ✗"] T5["Cadaveric: doctors\ndelivered in Ward 1,\nmidwives in Ward 2;\nKolletschka death confirms\n→ explains contrast ✓"] end FACT --> CONTRAST CONTRAST --> H1 & H2 & H3 & H4 & H5 H1 --> T1 H2 --> T2 H3 --> T3 H4 --> T4 H5 --> T5 INFER["INFER: Cadaveric infection\nbest explains the contrast\n→ handwashing intervention"] T5 -->|"loveliest:\nexplains all contrasts\nSemmelweis created"| INFER classDef fact fill:#1e3a5f,stroke:#5b9bd5,color:#fff classDef hyp fill:#2d1b69,stroke:#9d4edd,color:#fff classDef fail fill:#4a1942,stroke:#e94560,color:#aaa classDef pass fill:#1a4a2a,stroke:#52b788,color:#fff classDef infer fill:#1a3a2a,stroke:#52b788,color:#fff,stroke-width:3px class F1,P,Q,PQ fact class H1,H2,H3,H4,H5 hyp class T1,T2,T3,T4 fail class T5 pass class INFER infer ``` > The contrastive structure is philosophically significant: Semmelweis is not explaining *why anyone got fever* but *why Ward 1 got fever at a higher rate than Ward 2*. Switching the contrast class changes what counts as an explanation. A good explanation of P-rather-than-Q must cite a difference between the P-case and the Q-case — which is exactly what the cadaveric infection hypothesis does and the atmospheric hypothesis does not. --- *See also: [[Lipton - Ch05 Contrastive Inference]] · [[Lipton - Ch08 Explanation as a Guide to Inference]] · [[Lipton - Ch09 Loveliness and Truth]] · [[Lipton - Ch11 Truth and Explanation]]*