- Introduction - One or two paragraphs max - Define your terms and say what position you are going to defend. - 1. State [[the problem]] - Do we choose a view that allows for proper/accidental function, or do we choose one that can't cope with novel artifacts? - 2. Explain why we should not give up proper/accidental function - 3. Explain why we should give up [[novel prototype]] artifacts - Conclusion ## Slide: "Ethics in Design: Overview" **[[Speaker's Notes]]:** - **This is the uncomfortable part of the course** - Unlike aesthetics (subjective beauty debates) or function (testable outcomes) - Ethics asks: _Should this thing exist at all?_ - Forces designers to confront their role as decision-makers about society - **"Material and virtual environment" is not abstract** - Alarm clock that wakes you (or doesn't) - first designed interaction of your day - Phone interface you check within minutes - shapes attention patterns - Architecture determining how you move through space - Average person interacts with hundreds of [[designed objects]] daily - Most interactions are unreflective - we don't notice we're being shaped - **Key concept: "mediation" from [[philosophy of technology]]** - Design doesn't just provide tools - it transforms human capabilities - Google Maps example: [[not just]] directions, but mediating relationship with physical space - Students probably can't read paper maps anymore - that's not neutral - This is engineered obsolescence of human skills - Mediation is often invisible - most powerful when unnoticed - **Three connections form a progression (increasingly radical)** - First: straightforward rule-following (don't design toy-like landmines) - Second: harder - choosing _what_ to create (needs vs manufactured wants) - Third: most radical - does Design require reconceiving ethics itself? - Each level challenges designer autonomy and responsibility more deeply - **Historical context students should know** - Modernist designers believed functional design = ethical design - This chapter systematically dismantles that comfortable assumption - We'll see this optimism can't be maintained > “Design is relevant to ethics in both senses, but particularly in the broader sense, given that it has a great effect upon how we live through its creation of our material, and, increasingly, our immaterial or virtual, environment. As we have seen, this ‘mediation’ of our experience through technology occurs not only when Design makes a new kind of deliberate action possible, but also through the more subtle ways that Designs shape our unreflective behavior and thinking.” --- ## Slide: "Understanding Ethics" **[[Speaker's Notes]]:** - **Everyday vs philosophical ethics distinction is crucial** - Everyday: "ethical person" = rule-follower (doesn't lie, cheat, steal) - This is the narrow, compliance-based understanding - Think of it as "checking boxes" on moral behavior - **Philosophical sense is much broader and more demanding** - [[Central question]]: "How should a human being live?" - [[Not just]] "what rules do I follow?" but "what makes a life worth living?" - Includes: what kind of person to become, which activities have value - Much harder because no rulebook can cover this - **Why Design relates to broader sense particularly** - Design shapes the context in which we live [[our lives]] - Creates possibilities and forecloses others - Example: Car-centric design determines we spend hours daily in traffic - This isn't about individual actions but about structured ways of living - Designers are therefore implicated in "the [[good life]]" question - **Concrete example to ground this** - Narrow ethics: "Don't design an unsafe staircase" - Broad ethics: "Should we design cities around cars or public transit?" - Second question implicates visions of community, environment, human flourishing - No code or regulation can answer this - requires value judgments - **Challenge for students to consider** - Most Design training focuses on narrow ethics (safety codes, regulations) - But most consequential Design decisions involve broad ethics - Training doesn't match responsibility > “In philosophical contexts, however, the term is used a broader way to refer to a concern with [[the question]] of how one should live, or what a good human life consists in. This broader sense of the term incorporates the narrower sense, since rules or norms that constrain how we should treat others are one important aspect of the way we live. But it also includes much more, and relates to questions about [[what sort]] of person we should strive to be, which activities are worthwhile and which are not, and so on.” --- ## Slide: "Unethical Design" **[[Speaker's Notes]]:** - **Clear cases are thankfully rare but instructive** - Explosive devices designed to appeal to children = unambiguous - These cases reveal what we all agree on: malicious intent is wrong - But also show how limited pure intentionality is as criterion - **More common: negligence rather than malice** - Cost-cutting that compromises safety - Ford Pinto case (1970s): gas tank placement made car explode in rear-end collisions - Ford calculated wrongful death lawsuits would cost less than recall - This is evil through spreadsheet, not through intention - Students should know: most unethical design is economically rational - **Handguns example introduces crucial ambiguity** - NRA slogan: "Guns don't [[kill people]], people kill people" - Implies problem is misuse only, object itself is neutral - Counter-argument: standard use (concealed carry) facilitates violence - Even "responsible gun ownership" enables gun theft, accidents - Question: Can an object be unethical even when used as intended? - **Why this matters for designers** - Can't hide behind "I just made the thing, not responsible for use" - Use is often predictable, even if not universal - Designing cigarettes means knowing they'll kill users who follow instructions - Designing addictive social media means knowing psychological harm - **Recent example students will know** - Smartphone app selling restaurant reservations from fake bookings - Clogs restaurants with blocks of phony reservations, sold at profit - Function works perfectly; business model is exploitative - Many view as inherently unethical - system only works through harm > “It is beyond dispute that to create certain kinds of products, or to Design them in certain ways, would be unethical. To Design an explosive device in bright colors that would appeal to children, for example, would very clearly be wrong… One can argue that it is not the production, or use, of the object per se that threatens ethical norms, but only certain misuses of the object. Hence the infamous slogan of the National Rifle Association, that ‘Guns don't kill people, people kill people.’” --- ## Slide: "Three Subdivisions of Ethics" **Speaker's Notes:** - **Meta-ethics: the foundations** - Most abstract: "What even _is_ a moral norm?" - "Where do moral judgments come from?" - Are they objective facts, social constructions, or expressions of emotion? - Designers rarely work here, but assumptions matter - Example: If morality is just cultural preference, whose culture wins? - **Normative ethics: the frameworks** - This is the "big theories" level - Utilitarianism (Mill): maximize happiness/minimize suffering - Deontology (Kant): follow universal rules regardless of consequences - Virtue ethics (Aristotle): cultivate excellent character - Each gives different answers to same design problems - **Utilitarian approach to design** - Calculate total welfare: benefits minus harms - Self-driving cars that kill fewer people overall = good - Problem: might require sacrificing individuals for greater good - "Would you design a car that kills the driver to save five pedestrians?" - **Deontological approach to design** - Some things are just wrong regardless of outcomes - Manipulating users through dark patterns = wrong even if it "works" - Problem: can be rigid, impractical - "Never lie" is hard rule when designing privacy-protecting systems - **Applied ethics: where Design lives** - Specific actions and products: "Is _this_ design ethically permissible?" - Abortion, euthanasia, genetic engineering - particular cases - Design questions: "Should I design this surveillance tool?" - "Is it okay to design addictive games?" - **Why subdivision matters** - Different levels require different expertise - Designer might handle applied ethics (this specific product) - But deeper questions require philosophical training they don't have - This creates the "invisible tyrant" problem we'll discuss later > “Meta-ethics is the branch of philosophy that attempts to understand, in a very general way, the nature of ethics itself… Normative ethics… attempts to spell out the most general principles or frameworks for ethical decision-making… Finally, applied or practical ethics seeks to understand particular moral problems and determine whether particular actions… are morally permissible or not.” --- ## Slide: "Technological Determinism" **Speaker's Notes:** - **The determinist view: technology has its own logic** - Not controlled by human decisions but by internal development - "Whatever can be made, eventually will be made" - Technology shapes society; society doesn't shape technology - Determinism makes ethics seem pointless: resistance is futile - **Why this view is seductive** - Explains why dystopian tech gets built despite objections - Social media knew about mental health harms, built it anyway - AI development continues despite existential risk warnings - Feels like we're on a train with no brakes - **Feng's counter-argument: Moore's Law example** - Microprocessors double in power every 18-24 months - Seems like pure technical progression, inevitable - BUT: requires massive social investment in R&D - Billions of dollars in funding, thousands of researchers - Society could stop funding this; we choose not to - "Inevitability" is actually sustained by continuous choice - **Even if determinism partially true, ethics still matters** - _What_ gets made may have some inevitability - _How_ it gets designed is completely open - Photocopier example is brilliant in its simplicity - **Photocopier default setting example** - One-sided vs two-sided printing default - Seems trivial: one button setting - But: most users never change defaults - One-sided default = massive paper consumption - Two-sided default = immediate environmental benefit - This choice has larger impact than most "green" initiatives - Designer made this choice, possibly without thinking about it - **Implications for student designers** - Can't excuse yourself with "technology is inevitable" - Can't say "I'm just following market forces" - Hundreds of micro-decisions in every project - Each one is an ethical choice, even if unrecognized > “Technological determinism is the idea that the development of technology is not controlled by human thought and decision-making… whatever can be made, given current scientific and technical knowledge, eventually will be made… Yet, Feng observes, the development would not occur were it not for heavy investment in processing research… Does one make the default setting ‘one-sided printing’ or ‘two-sided printing?’ While the choice may seem trivial, it could have enormous implications for paper use and thereby for environmental sustainability.” --- ## Slide: "Complexity of Design Ethics" **Speaker's Notes:** - **Safety codes aren't sufficient - here's why** - Legal standards exist for _known_ product categories - Toasters, cars, construction - well-established codes - But design is fundamentally about novelty - New products lack applicable codes by definition - Takes years for regulation to catch up - **Historical examples of code lag** - Early social media: no regulations about data collection - Cryptocurrency: still largely unregulated - CRISPR gene editing: ethics scrambling to catch up - Designers were making crucial decisions in regulatory vacuum - **Multiple ethical values beyond safety** - Freedom: Can users opt out? Are they trapped? - Availability: Who can access this? Just the wealthy? - Consumer choice: Does this create lock-in effects? - Privacy: What data is collected, how is it used? - Environmental sustainability: Full lifecycle impact? - **Design affects ability to realize all these values** - Not just "does this hurt users?" but "what life does this enable?" - iPhone design: incredibly safe, but creates digital dependency - Suburban design: very safe (low crime), but eliminates walkability - Amazon interface: maximizes choice, but destroys local retail - Each design choice privileges some values over others - **Why this is overwhelming for designers** - Engineering training focuses on safety and function - Little training in value conflicts and philosophical analysis - Yet expected to balance all these considerations - No clear methodology for resolution - **Real-world complexity example** - Design a smart home system - Convenience vs privacy (cameras everywhere) - Security vs freedom (can't turn off some features) - Efficiency vs jobs (replaces human labor) - Accessibility vs cost (expensive systems exclude poor) - Every feature implicates multiple conflicting values > “However, the nature of Design as a creative activity means that, in some cases at least, things will not be so straightforward, since there simply will not be extant codes and regulations covering the new object… Others include freedom, availability, consumer choice, privacy and environmental sustainability… The applied ethics of Design can involve all of these values.” --- ## Slide: "Conflicting Values in Design" **Speaker's Notes:** - **Core insight: highest quality ≠ most ethical** - Intuition says: "Make it as good as possible" - But "good at its function" can be "bad for the world" - Quality is narrow; ethics is broad - This undermines entire Modernist project (more on this later) - **Function vs environmental sustainability** - Design perfectly functional product that consumes massive resources - Incandescent bulbs: excellent light quality, terrible efficiency - Crypto mining computers: work perfectly, environmental disaster - Fast fashion: clothes function fine, ecological catastrophe - Functionality and sustainability often oppose each other - **Efficiency vs consumer choice** - Making something efficient often makes it expensive - High-quality, durable goods cost more upfront - Eliminates options for people with less money - Planned obsolescence is democratizing (in perverse way) - "The poor can't afford cheap things" paradox - **Function vs user freedom: seatbelt-ignition example** - Safety feature: car won't start without seatbelt fastened - Functionally perfect: dramatically reduces accident deaths - But: removes user choice, even for legitimate reasons - What if driving 10 feet in driveway to wash car? - What if medical condition makes seatbelt dangerous? - Every safety feature is a freedom reduction - **Security vs privacy: the eternal tradeoff** - Effective threat detection requires surveillance - Airport security: very safe, very intrusive - Contact tracing apps during pandemic: health vs privacy - Encrypted messaging: privacy vs law enforcement - No technical solution makes both values maximal - **Why these conflicts are philosophically interesting** - Can't be resolved through more information - Can't be solved through better engineering - Require value judgments about what matters more - Different ethical frameworks give different answers - This is where design becomes unavoidably political - **Designer's impossible position** - Must make choice even when no right answer exists - Choice gets embedded in material culture - Affects potentially millions of people - Often irreversible once deployed at scale - And designer probably has no special ethical training > “From the point of view of functionality, it seems that Designers should always strive for the highest-quality goods possible. But… this might not always result in realizing certain ethical values… The operation of a perfectly functional Design might also, through its normal operation, compromise user freedom, in the way that seatbelt-linked ignition systems do. Security systems that are very effective… can compromise privacy.” --- ## Slide: Discussion - Technological Determinism **Speaker's Notes:** - **This question is deliberately provocative** - Challenges whether designers have real agency - "Elaborate fiction writers" suggests it's all pretense - Market forces and path dependencies as real determinants - **Path dependency concept** - QWERTY keyboard: designed to slow typists (prevent jamming) - Now completely unnecessary, but locked in - Better layouts exist (Dvorak), but switching costs too high - Past choices constrain present options - **Market forces as hidden author** - "Design what sells" vs "design what's needed" - Quarterly earnings pressure drives design decisions - "Disruptive innovation" often just means profitable - Designer might want sustainable design, market demands cheap - **Points to raise in discussion** - Even if constrained, designers make choices within constraints - Photocopier example: zero cost difference, huge impact - "I was just following orders" is not ethical defense - Perhaps designers need to refuse certain projects - **Counter-argument to explore** - Maybe determinism is excuse for avoiding responsibility - "Nothing I can do" is convenient abdication - Individual designers at Facebook made concrete choices - Those choices weren't inevitable, even if profitable - **Connect to broader themes** - This is ancient question of free will in new context - If we're all determined, does ethics disappear? - Chapter will argue no: ethics focuses on human decision-making - Even if constrained, still meaningful choices > “Even if there were a kind of inevitability about what things will be produced in the future, there are still many ethically important choices to be made about how those things will be Designed.” --- ## Slide: Discussion - Safety Features and Authoritarianism **Speaker's Notes:** - **This is one of the best questions in the chapter** - Reveals ratchet mechanism in design ethics - We can only add restrictions, never remove them - "Politically impossible" to remove safety features - **Historical examples of this ratchet** - Once seatbelt laws passed, can't repeal without seeming pro-death - TSA security theater: everyone knows it's ineffective, can't be cut - Child safety features on medicine bottles (harm elderly who can't open them) - Each safety measure becomes permanent regardless of cost/benefit - **The libertarian objection** - Every safety feature is paternalism - Treats adults like children who can't assess risk - Motorcycle helmet laws: protecting people from themselves - At what point is safety coercion? - **The public health counter-argument** - Individual risk becomes societal cost (healthcare, productivity loss) - Your "freedom" to not wear seatbelt affects others - Externalities justify paternalism - But: slippery slope to controlling all behavior - **"Principled stopping point" challenge** - Is there a design principle that says "this much safety, no more"? - Utilitarian calculation: diminishing returns on safety - But: can't publicly say "acceptable death rate" - Politicians won't say "this feature costs $1M per life saved, not worth it" - **Design ethics as inherently authoritarian?** - If we can only add, never subtract, is this just slow authoritarianism? - 100 years from now: layer upon layer of restrictions - Death by thousand papercuts to human freedom - Or: this is what civilization is - accumulated wisdom about danger - **Questions to pose to students** - Should designers resist client demands for safety features? - Is there such thing as "too safe"? - Who decides optimal freedom/safety tradeoff? - Can design preserve user agency while protecting from harm? > “The operation of a perfectly functional Design might also, through its normal operation, compromise user freedom, in the way that seatbelt-linked ignition systems do. Security systems that are very effective in detecting potential threats to public safety can compromise privacy.” --- ## Slide: Discussion - Social Media Feature Harm **Speaker's Notes:** - **This scenario is not hypothetical** - Facebook's own research showed Instagram harms teen mental health - Engagement features worked exactly as designed - "Success" metric (engagement) misaligned with human welfare - Company knew, continued anyway, internal docs leaked - **Key philosophical puzzle: harm was in the brief** - You did exactly what you were asked to do - Feature performed perfectly (15% engagement increase) - Designer followed instructions competently - Yet harm resulted - who's responsible? - **Traditional ethics says: intent matters** - You didn't intend harm (intended engagement) - Harm was unforeseeable (though maybe should have been foreseen) - This suggests designer is not culpable - But: does this let designers off hook too easily? - **Consequentialist ethics says: outcomes matter** - Intentions are irrelevant, only results count - Teen depression and self-harm are real - Designer causally contributed to this harm - Therefore designer shares responsibility - **The "just following orders" problem** - Nuremberg defense: "I was just doing my job" - We rejected this in Nazi war crimes trials - Set precedent: following orders is not moral excuse - Does this apply to designers? Should it? - **Who should be punished question** - Designer who implemented feature? - Manager who wrote the brief? - Executive who approved it? - Shareholders who profited? - Users who engaged with feature? - Distributed responsibility means no one responsible? - **Structural problem revealed** - Misaligned incentives at every level - Designer evaluated on engagement, not welfare - Company valued on growth, not social impact - Market rewards addiction, punishes restraint - Individual ethics can't fix structural problems - **Questions for students** - Would you implement this feature if you knew harms? - Would you implement without knowing, without asking? - At what point do you have duty to research potential harms? - Is resignation your only ethical option? - Does staying and trying to change from within work? > “The question confronting the Designer is not simply the empirical question – Will this device do what it is supposed to? – but rather the ethical question – Should there be a device that does this?” --- ## Slide: "Modernism and Ethical Design" **Speaker's Notes:** - **Modernist claim: functional design = ethical design** - This was a beautiful, optimistic vision - Strip away ornament, focus on function, ethics follows - Form follows function → truth follows function → goodness follows function - Seductive because it makes ethics seem automatic - **Why this can't be maintained broadly** - We just discussed how function can conflict with freedom, privacy, etc. - Highly functional designs can be ethical disasters - Functional surveillance is still surveillance - Functional addiction is still addiction - No automatic link between "works well" and "is good" - **More modest Modernist claim worth considering** - Maybe functional design combats wasteful consumerism - Stop redesigning for aesthetic/expressive reasons - Save resources by designing once, designing well - **Automobile example: annual redesign** - Cars redesigned yearly with superficial changes - Tailfins, chrome, styling changes - Functionally unnecessary, pure marketing - Generates enormous consumption through manufactured obsolescence - Fashion applied to durable goods - **This modest claim seems reasonable until...** - What about functional design for pointless purposes? - Can design something that works perfectly for stupid purpose - This reveals function alone is insufficient - **Leaf blower: the perfect example** - Leaf blower may be excellently designed - Highly functional: blows leaves efficiently - Powerful motor, ergonomic, reliable - Problem: the function itself is gratuitous - **Why leaf blower is gratuitous** - Can rake leaves into pile - Raking is quiet, good exercise, no emissions - Leaf blower is loud, polluting, annoying to neighbors - Saves perhaps 15 minutes, generates noise pollution for hours - Pure preference for technological solution over manual one - **Broader implication** - Functional excellence for unnecessary purposes is waste - This is distinctively modern problem - Only wealthy, technological societies design leaf blowers - Must distinguish necessary from unnecessary functions - This requires need/want distinction - **Modernist position revised** - Not: functional design is ethical design - But: functional design for genuine needs is ethical design - This shifts burden to: what are genuine needs? - Next sections tackle this question > “Clearly this thesis cannot be maintained, if we consider the ethical values relevant to Design in the broadest sense… A well-designed leaf blower might be very effective at carrying out its function: blowing leaves around. The problem with devices like this one is not that they are created for aesthetic or expressive reasons, but that the functions they fulfill are, in some sense, gratuitous or unnecessary. There is no need to blow leaves around, given that one can rake them into a pile.” --- ## Slide: "The Want/Need Distinction" **Speaker's Notes:** - **Casual language obscures real distinction** - People say "I need coffee" when they mean "I want coffee" - Product reviews: "Do you need this gadget?" (really: "will you want it?") - Marketing deliberately conflates the terms - But there IS a real distinction in our linguistic practice - **First evidence: wants without needs** - Third pair of red running shoes - Fifth streaming service subscription - Novelty gadgets (electric wine opener, etc.) - We recognize these as wants-not-needs - **Second evidence: needs without wants** - Child needs education but doesn't want homework - Patient needs surgery but doesn't want pain - Addict needs intervention but doesn't want it - "Need" can exist independently of desire - **This shows they're conceptually distinct** - Want: subjective psychological state - Need: objective requirement based on life situation - Want is about me; need is about my circumstances - **Objectivity of needs is crucial** - Want something = I have certain relation to it - Need something = thing is "called for" by real features of situation - Needs make claims on others in way wants don't - This grounds the "Principle of Preference" (next slide) - **Implications for design** - If needs are objective, they can be wrong about - People can think they need things they don't - Advertising industry exploits this: creates false needs - Designer can't just ask users "what do you need?" - Must make independent judgment about genuine needs - **This sets up fundamental tension** - Democratic impulse: give people what they want - Paternalistic impulse: give people what they need - Designer caught between respecting autonomy and promoting welfare - No easy resolution to this tension > “This distinction shows itself in several ways, firstly in the fact that we often acknowledge that people want things that they don't need… and that, conversely, people need things that they don't want… For someone to need something… the thing in question must be in some way ‘called for’ by real features of his life situation.” --- ## Slide: "The Principle of Preference" **Speaker's Notes:** - **Frankfurt's insight about moral force** - "I need X" creates moral claim on others - "I want X" creates no such claim - This is intuitive but needs examination - **Espresso machine vs food example** - Person A wants new espresso machine very intensely - Person B wants food enough to stay alive (maybe less intensely due to despair) - We give precedence to B automatically - Not based on intensity of desire but on need vs want - **Why this is philosophically interesting** - Challenges simple utilitarian calculation - Not just "maximize satisfaction of desires" - Must weight needs more heavily than wants - But how much more? Infinitely? Proportionally? - **Real-world application: resource allocation** - Hospital triage: needs-based, not wants-based - Food aid: goes to malnourished, not gourmets wanting variety - Housing policy: shelter as need, luxury apartments as wants - But: fuzzy boundary cases everywhere - **The wealthy person's paradox** - Billionaire who "needs" fifth yacht for tax purposes - Genuinely part of their financial planning - But we don't grant this moral priority - Shows "need for larger end" isn't sufficient (discussed later) - **Implications for design practice** - Should designers prioritize projects addressing needs? - Morally superior to design affordable housing vs luxury condos? - Design water purification vs designer coffee makers? - Market rewards wants-based design more highly - **The designer's dilemma** - Know you should design for needs - But employment depends on designing for wants - Luxury goods design pays better than humanitarian design - Structural problem: ethics and economics misaligned - **Questions to raise** - Is all wants-based design morally problematic? - What about art, entertainment, pleasure? - These address wants, not needs, but seem valuable - Maybe needs/wants distinction too crude > “ ‘An assertion that something is needed tends to create an impression of an altogether different quality, and to have a substantially greater moral impact, than an assertion that something is desired’ (Frankfurt)… If one person desires a new espresso machine, and another desires sufficient food to keep himself alive, we naturally give precedence to satisfying the second person.” --- ## Slide: "Beyond Basic Needs" **Speaker's Notes:** - **Basic needs are uncontroversial** - Food, water, air, shelter - Necessary for biological survival - Everyone agrees these are genuine needs - Objective: without them, you die - **But basic needs are far too narrow for design** - Would restrict design to architecture, agriculture, medicine - No electricity, no indoor plumbing, no transportation - Our ancestors survived without these - Modern life would be impossible - **The misery argument** - "You'd survive" is not good enough - Medieval life expectancy: 30-40 years - High infant mortality, constant pain, limited options - Survival ≠ flourishing - Need should relate to good life, not just biological existence - **Design examples beyond basic needs** - Telephones: not required for survival - Leaf blowers: also not required for survival - But intuitively, telephone seems more like genuine need - Why? What's the difference? - **Telecommunications intuition** - Modern employment requires phone access - Emergency services require communication - Social connection requires ability to reach others - "Need" phone in sense: required for full participation in society - **Leaf blower intuition** - Not required for employment, safety, or connection - Purely convenience device - Minimal time savings, maximum annoyance - "Want" in sense: nice to have, not necessary for anything important - **This reveals what we're looking for** - Not just biological survival - But requirements for social participation, human flourishing - Needs are socially and historically situated - Medieval person didn't need phone; modern person does - **Relativism worry** - If needs are socially situated, are they just conventional? - Does "need" collapse into "what my society expects"? - Then no critical distance from consumer culture - Can't critique false needs if needs are whatever society says - **This motivates the search** - Need broader conception than basic needs - But objective enough to critique consumer culture - Next slide explores "needs as means to ends" > “ ‘Things necessary for survival’ is a very narrow conception of need… While this would no doubt solve the ethical problem of over- consumption, it is too extreme to merit serious consideration… while many Design objects address aims other than the basic needs, they are not thereby unnecessary or gratuitous… neither telephones nor leaf blowers are required for biological survival but, intuitively, the second is a much clearer example of ‘unnecessary design’ than the first.” --- ## Slide: "Needs as Means to Ends" **Speaker's Notes:** - **Frankfurt's criterion: "specify what it's needed for"** - Intuitive idea: needs are instrumental, wants are terminal - Need something for something else - Want something for its own sake - Seems to capture the distinction - **Car-for-work example** - Need car to get to job - Need job to earn money - Need money to live - Chain of instrumental relations - This looks like genuine need - **Convertible "just to have it" example** - No larger purpose specified - Desired for its own sake (or status signaling) - Terminal desire, not instrumental - This looks like mere want - **This approach seems promising until...** - You realize any want can specify a "larger end" - Rationalization is easy, perhaps too easy - **Third convertible example** - "I need it to complete my collection" - There IS a larger end: collection completion - Collection is genuine project spanning years - Requires this specific car to achieve - So... is it a need? - **Intuitively, no - but why not?** - Frankfurt's criterion seems satisfied - Yet we want to say this is false need - Shows the criterion is insufficient - Any want can be embedded in larger project - **Further examples of rationalized wants** - "Need" designer clothes to maintain professional image - "Need" expensive car for client meetings - "Need" luxury vacation to avoid burnout - "Need" latest iPhone to stay current with technology - All specify larger ends, all feel like rationalizations - **What's gone wrong with this approach?** - The "larger end" itself might be unnecessary - Convertible collection is not necessary project - Regresses: need X for Y, but do you need Y? - Eventually hits arbitrary life projects - **Implication: infinite regress problem** - Keep asking "but why do you need that?" - Eventually reach choices, not necessities - Everything ultimately grounds in arbitrary preferences - This threatens to make all needs (beyond basic) subjective - **This motivates skeptical view (next section)** - Maybe needs beyond basic needs are just wants in disguise - The concept is doing no real work - Just making our wants feel justified > “One natural strategy here would be to understand a need as something that is not desired merely for its own sake, but is required to achieve some larger end… ‘when something is needed it must ... be possible to specify what it is needed for’ (Frankfurt)… Generally speaking, it seems that any want can be cast in the form of a need, using this criterion.” --- ## Slide: Discussion - Functional Need-Based City **Speaker's Notes:** - **This thought experiment is deliberately dystopian** - Take needs-based design to logical extreme - What would perfectly rational, need-based city look like? - Housing allocated by proximity to work (need to minimize commute) - Private vehicles banned (needs public transit efficiency) - Parks eliminated (don't need recreational space for survival) - **Why this is dystopian** - Sounds like Soviet rationalization of scarcity - Eliminates human spontaneity, choice, pleasure - Reduces humans to economic functions - Yet follows directly from "design for needs, not wants" - **What's missing from the framework?** - Human flourishing includes wants, not just needs - Play, beauty, spontaneity are not needs but are valuable - Efficiency is not the only value - Room for "waste" is essential to good life - **The park example is instructive** - Parks seem unnecessary: don't need for survival - Could use that space for housing, production - But parks serve human flourishing in deep ways - Mental health, community gathering, contact with nature - Are these needs or wants? - **Blurry boundary problem** - Mental health is genuine need - Parks serve mental health - Therefore parks address needs? - Or: parks serve want for beauty, and that's okay? - **The aesthetic need question** - Do humans need beauty? - Maslow's hierarchy: aesthetics near top, not bottom - Suggests luxury, not necessity - But: humans in all cultures create art - Perhaps aesthetic experience is human need - **Connects to Modernism** - Modernists tried to unify function and beauty - "Form follows function" was supposed to generate beauty - This dystopia shows function alone is sterile - Need aesthetic dimension as well - **Questions for discussion** - Where should designers draw the line? - Is some "waste" ethically obligatory? - Can we have principled way to include wants? - Or is needs/wants framework fundamentally flawed? > “ ‘Things necessary for survival’ is a very narrow conception of need… While this would no doubt solve the ethical problem of over- consumption, it is too extreme to merit serious consideration.” --- ## Slide: Discussion - Prosthetics vs Universal Design **Speaker's Notes:** - **This is a brilliant dilemma for utilitarian thinking** - Prosthetics for landmine victims: clear good, obvious need - Universal design features (ramps, better interfaces): prevents frustrations - Both improve welfare, but which project to pursue? - **Prosthetics case is emotionally compelling** - Identifiable victim (vs statistical lives) - Dramatic need (vs minor inconveniences) - Visible impact (person can walk again) - Easy to photograph, fundraise for - Heroic narrative: designer as savior - **Universal design case has larger impact** - Millions of people, multiple times daily - Cumulative frustration adds up - Broader accessibility benefits - Prevents problems rather than fixing them - But: no dramatic narrative, hard to visualize - **Utilitarian calculation** - Sum total welfare improvements - Universal design probably wins on pure numbers - 10 million people × small improvement > 100 people × large improvement - But: diminishing marginal utility complicates this - **Does needs framework help?** - Landmine victim has clear, urgent need - Daily frustrations are mere inconveniences (wants?) - Needs framework privileges prosthetics - But is this right answer or just emotionally satisfying answer? - **Chapter's suggestion: needs framework rationalizes emotion** - We want to help identifiable victim - "Needs" language makes this feel principled - Actually just following bias toward visible, dramatic cases - Needs framework obscures rather than clarifies - **Effective altruism critique** - Most "meaningful" feeling work is inefficient - Helping identifiable victims feels good - Preventing statistical harms is more effective - We should do less satisfying but more impactful work - **Counter-critique of effective altruism** - Reduces human welfare to fungible units - Ignores special obligations, personal relationships - Loses sight of human meaning and narrative - Pure calculation is inhuman - **Designer caught in middle** - Both projects seem worthwhile - Limited time and resources - Must choose - Needs framework doesn't resolve choice, just makes one option feel better - **Questions to explore** - Should designers trust their emotional responses? - Is there wisdom in caring more about visible victims? - Or is this bias we should overcome? - Can design serve both needs and broader welfare? > “The core of Aristotle's account is that needs are those things required for us to achieve things of value (things that are good, or worthwhile).” --- ## Slide: "The Sceptical Challenge" **Speaker's Notes:** - **Skeptic's position: only basic needs are real** - Food, water, shelter, air - these are objective - Everything else is just intense desire with fancy label - "Need" beyond basics is self-deception - **False needs as self-justification** - People don't want to admit they're being selfish or wasteful - Saying "I need" feels more justified than "I want" - Example: "I need a new suit" vs "I want a new suit" - First sounds legitimate, second sounds frivolous - But meaning is the same - **This view has real force** - We do use "need" to rationalize desires - Consumer culture encourages this - "Treat yourself - you deserve it" = "you need it" - Marketing creates sense of necessity - **If skeptic is right, implications are severe** - Can't distinguish necessary from unnecessary design - Can't critique consumer culture from within - Modernist project (design for needs) collapses - All design becomes equally arbitrary - **This threatens the entire ethical framework** - Can't reduce consumption by designing for needs - Because "needs" (beyond basic) don't exist - Just varying intensities of desire - Should design for most intense desires? Most common? Most profitable? - **Why take skepticism seriously** - Previous attempts to define needs failed - "Needed for larger end" collapsed into rationalization - Social/historical relativity threatens objectivity - Maybe we should bite the bullet - **But skepticism also seems wrong** - Education really does seem like need, not want - Healthcare seems like need - Social connection seems like need - These aren't just intense desires - **The challenge: explain what makes them needs** - Can't be intensity (might want convertible more intensely) - Can't be larger end (convertible serves collection) - Can't be social convention (too relativistic) - Need new approach - this sets up Aristotle > “The skeptical view of needs as no more than intense desires… If it is true, then it seems to throw into doubt the thesis that we can reduce consumption by Designing for needs… Here the skeptic may say that, aside from the basic needs, the ‘needs’ we speak of are false needs: nothing more than intense desires dressed up in ‘objective’ garb.” --- ## Slide: "Origins Don't Determine False Needs" **Speaker's Notes:** - **Papanek's view represents common intuition** - Real needs are natural, spontaneous - False needs come from external manipulation - Advertising "engineers" and "inculcates" wants - Fashion and fad create false needs - **This view is appealing because** - Advertising obviously creates desires - Multi-billion dollar industry exists to manufacture wants - People desire things they never heard of before ad campaign - Planned obsolescence creates "need" for upgrades - **Examples of advertising-created needs** - Mouthwash: Listerine invented "halitosis" as medical condition - Diamond engagement rings: De Beers marketing campaign - Yearly iPhone upgrades: minimal improvements, massive desire - These seem like paradigm cases of false needs - **But origins criterion fails on analysis** - Medical conditions often require external diagnosis - Person might not know they have diabetes until doctor tells them - Doctor's diagnosis involves persuasion ("you need treatment") - This is external influence, but need is real - **Education example is even clearer** - Ignorant person doesn't know they need education - Literally can't recognize their own ignorance (Dunning-Kruger) - Teacher must persuade them education is necessary - May involve significant persuasion, even coercion - But this doesn't make education a false need - **The parent-child case** - Child doesn't want vegetables, wants candy - Parent must persuade/coerce child to eat well - External influence is total - But nutritional needs are real - **So external influence is not the problem** - Both true and false needs can come from external influence - Origin doesn't determine authenticity - Must look elsewhere for distinction - **This shifts focus to value** - Not where need comes from - But whether thing "needed" is genuinely valuable - Education is valuable; third convertible is not - But now must defend value judgments - This is harder but more honest approach > “Might we not say that false needs are needs that we would fail to recognize, were it not for the influence of some external force, such as advertising?… However, it would be a mistake to understand false needs as those with a particular origin… A person with a medical condition… might never come to recognize that they need medical treatment or education without being influenced by some outside agency… But this would not make their needs any less real.” --- ## Slide: "Aristotle's Account of Need" **Speaker's Notes:** - **Aristotle's two-part definition** - First: things necessary for survival (basic needs) - covered already - Second: things necessary "for good to exist or come to be" - Second criterion is the breakthrough - **"For good to exist or come to be" explained** - Not just biological survival - But achieving things of value, worthwhile things - Avoiding bad things, attaining good things - This grounds needs in value, not just desire - **Why this solves previous problems** - Third convertible: collection is not genuinely valuable - Education: knowledge and capability are genuinely valuable - Healthcare: health is genuinely valuable - Distinction tracks our intuitions - **The value question is now explicit** - Must ask: is this thing valuable/worthwhile? - Not: do people want it? (too subjective) - Not: is it socially conventional? (too relativistic) - But: is it objectively valuable? - **Examples of applying Aristotle's criterion** - Need education: enables human flourishing, meaningful work, understanding - Need healthcare: health is precondition for pursuing any goals - Need social connection: humans are social animals, isolation harms - Don't need third convertible: collection is arbitrary project without real value - **The thinking-doesn't-make-it-so point is crucial** - Convertible collector thinks collection is valuable - May even think happiness depends on it - But this doesn't make it objectively valuable - Value is not just in the eye of beholder - **This is philosophically controversial claim** - Assumes value realism: some things really are more valuable than others - Rejects value relativism: value is not just preference or opinion - Most modern thinking tends toward relativism - Aristotle swims against contemporary current - **Historical context: Aristotle's ethics** - Part of eudaimonistic ethics: ethics of flourishing - Human beings have a telos (purpose/function) - Some activities realize our nature, others don't - Needs are what's required to realize our nature - **Modern resistance to this view** - Who's to say what's valuable? - Sounds elitist, paternalistic - But: some value judgments seem clearly correct - Health better than sickness, knowledge better than ignorance - Not all values are equally controversial - **Implications for design** - Designer must make value judgments - Not just "will people buy this?" - But "does this serve genuine human goods?" - This is enormous responsibility - Next slides explore whether designers can handle it > “Anything without which it is not possible for good to exist or come to be… The core of Aristotle's account is that needs are those things required for us to achieve things of value (things that are good, or worthwhile)… thinking that something is valuable does not make it so.” --- ## Slide: "The Designer's Burden" **Speaker's Notes:** - **The responsibility is now explicit** - If needs are based on value - And designers should design for needs - Then designers must make value judgments about good life - **This returns to ancient idea: Designer as philosopher** - Not just problem-solver or form-giver - But guide to human flourishing - Must understand what makes life worth living - Then embed that understanding in material culture - **Historical precedent: Plato's philosopher-kings** - Only those who understand the Good can rule - Rulers must be philosophers - Here: Designers must be ethicists - Similar epistemic requirement - **Campbell's objection: this is mere prejudice** - Who are you to say what's valuable? - Different people value different things - Declaring some needs "true" and others "false" is imposing your values - This is "prejudice in favor of one specific conception of the good life" - **Campbell's authoritarianism worry** - Leads to "traditionalistic, Puritanical... authoritarian attitudes" - History shows dangers of imposing conception of good life - Religious wars, communist central planning, fascist social engineering - Better to let people decide for themselves - **These objections have real force** - 20th century is littered with failed attempts to engineer good society - Paternalism often just means "my preferences imposed on you" - Reasonable disagreement about what's valuable - Liberal society respects pluralism - **But chapter argues objections are exaggerated** - Will respond on next slide > “If the Designer is to Design… only for need… the very decision to make at all implicates her in some kind of conceptualization of the way we ought to live… Thus we are brought back to the lingering idea of the Designer as a sort of seer or philosopher… The idea that Designers… can pronounce about what counts as ‘genuine need’ invites objections… Colin Campbell… argues that to declare some needs as true and others false is ‘simply to express a prejudice.’” --- ## Slide: "Value Claims Aren't Mere Prejudice" **Speaker's Notes:** - **First response: not all value claims are prejudice** - Some value claims are well-reasoned and grounded - Distinction between reasoned judgment and mere prejudice - Example: "Education is valuable" is not mere prejudice - Based on evidence about human flourishing - **Comparison to truth claims** - Easy to be dogmatic about what's true - But this doesn't mean all truth claims are dogmatic - Some beliefs are well-justified, others aren't - Same with value claims - **Second response: judgments ≠ authoritarianism** - Can make value judgments without imposing them - Free societies don't force people to accept particular values - Example: I can judge that smoking is bad without banning cigarettes - Liberal society compatible with value realism - **The live-and-let-live principle** - Making judgments ≠ forcing compliance - Can believe someone's life choices are unwise - While respecting their autonomy to make those choices - This is standard liberal position - **But design creates special problem** - Designer isn't just making arguments - Designer embeds values in material culture - This IS a form of imposition - **Why this is distinctive to design** - Philosopher writes book arguing for values - you can ignore it - Designer creates environment that shapes behavior - can't ignore it - Built environment determines what's possible, easy, difficult - This is power over others, whether intended or not - **Examples of designers imposing values** - Car-centric urban design imposes value of individual mobility over community - Open-plan offices impose value of collaboration over concentration - Smartphone interfaces impose value of constant connectivity - Each design choice privileges certain ways of living - **The "invisible tyrant" problem preview** - Designer makes value judgments - Embeds them in designs that shape millions of lives - Often subtly and unrecognizably - People don't even realize they're being shaped - This looks like covert authoritarianism - **No easy resolution** - Design must make choices - Every choice embeds values - But designer may lack legitimacy to make these choices - This tension drives the rest of the chapter > “It is also plainly not true that making judgements of value will lead to authoritarianism… [I]n societies where freedom is valued, judgements about the good will not be foisted upon individuals… However, Campbell's points do have a particular salience in the context of Design, for the Designer, unlike others, gets to embed her determinations of the ‘good life’ in material culture, and, in that sense, imposes them on others.” --- ## Slide: Question - Designer Imposing Values **Speaker's Notes:** - **This question captures the core paradox** - Chapter says: designers shouldn't impose values (invisible tyrant problem) - But also says: designers must distinguish true needs from false wants - Determining someone's want is "false" = imposing your value judgment - **The advertising critique** - False needs are "shaped by advertising" - Designer's role: resist advertising-created desires - Design for true needs instead - But who decides what's false vs true? - **Example: makeup and cosmetics** - Cosmetics industry creates beauty standards - Generates "need" for products to meet those standards - Is this false need created by marketing? - Or genuine need given social realities? - **Feminist perspectives differ** - One view: beauty standards are oppressive, needs are false - Other view: given that standards exist, need products to navigate them - Both views could be held by designers - Whichever they choose, they're imposing values - **The luxury goods question** - Are luxury goods always false needs? - Or do they serve genuine goods (beauty, craftsmanship, status)? - Status seeking: is this false need or genuine human drive? - Designer must decide - either way imposes value judgment - **Possible responses to explore** - Maybe some value judgments are less controversial (basic needs) - Designer should stick to uncontroversial cases - Problem: most interesting design questions are controversial - **Alternative: embrace the imposition** - Yes, design imposes values - this is unavoidable - Question is not whether but which values - Better to be explicit and deliberate than implicit and unconscious - At least then we can debate the choices - **Or: democratic solution (participatory design)** - Don't have designer decide alone - Involve stakeholders in value decisions - But this just shifts problem: which stakeholders? How much weight? - **This question has no clean answer** - Reveals deep tension in design ethics - Will recur in various forms throughout remaining slides > “The idea that Designers, or anyone for that matter, can pronounce about what counts as ‘genuine need’ invites objections… [Y]et… the Designer, unlike others, gets to embed her determinations of the ‘good life’ in material culture, and, in that sense, imposes them on others.” --- ## Slide: "Do Artefacts Have Politics?" **Speaker's Notes:** - **Winner's 1980 essay is landmark text** - One of most influential papers in philosophy of technology - Challenged common sense view about technology - Title question seems absurd at first - **The common view Winner challenges** - Only people have politics; things are neutral - Technology is tool that can be used for good or ill - Ethics concerns human actions, not objects themselves - "To discover virtues or evils in steel and plastic seems just plain wrong" - **This view seems reasonable** - Guns don't kill people, people kill people - Internet can be used for liberation or oppression - Depends on the user - Technology itself is neutral medium - **Winner's counterargument: Moses's overpasses** - NYC highway overpasses built in 1950s by Robert Moses - Deliberately built too low for buses to pass - Buses were primary transportation for poor Black New Yorkers - Result: prevented them from reaching certain beaches and parks - Segregation built into infrastructure - **Why this example is powerful** - If you only look at human actions, you miss the segregation - White New Yorkers might not be actively racist - But racist outcome is achieved through built environment - Overpasses do the racist work automatically - **The "politics of artifacts" concept** - Artifacts themselves can direct and shape decisions - Not just tools used by humans - But active participants in power relations - Material environment isn't neutral backdrop - **Further examples students might recognize** - Stairs vs ramps: exclude wheelchair users - Highway through Black neighborhood: urban renewal as removal - Voting machines with complicated interfaces: suppress certain voters - Facebook's algorithm: shapes what information people see - **Each case: human action + artifact working together** - Can't fully understand politics by looking at humans alone - Can't fully understand by looking at artifacts alone - Must analyze the sociotechnical system - **Implications for designers** - Your designs will have political effects - Whether you intend them or not - Whether you recognize them or not - Can't hide behind "I just make things, users decide how to use them" > “We all know that people have politics; things do not… [Yet] the highway overpasses built in New York City in the 1950s by Robert Moses… were deliberately built too low for buses to pass underneath, thereby preventing the poorer Black New Yorkers, who did not own cars, from reaching certain areas. In this case… there is a political aspect to the artefact itself.” --- ## Slide: "The Material Morality Thesis" **Speaker's Notes:** - **Latour extends Winner's political analysis to moral domain** - Not just politics but morality embedded in artifacts - Artifacts are "non-human moral agents" - We have "delegated" values, duties, ethics to them - **Latour's provocative claim** - "It is because of this morality that we, humans, behave so ethically" - Our ethical behavior depends on artifacts - Not: we use artifacts to act ethically - But: artifacts make us ethical - **Speed bump example** - Gets drivers to slow down - Much more reliable than moral exhortation - "Please drive slowly" signs don't work - Speed bump forces compliance - The artifact does moral work - **Other examples of material morality** - Seatbelt alarm: enforces safety - Childproof medicine caps: prevent poisoning - Breathalyzer ignition locks: prevent drunk driving - Airport security screening: prevents hijacking - **Verbeek's term: "material morality"** - Morality is not just in human minds and actions - But also in material arrangements - "Technologies play an active role in moral action and decision-making" - This is "moral agency of technological artefacts" - **The revolutionary claim** - Traditional ethics: only rational beings are moral agents - Agents must be capable of deliberation, free choice - This limits moral agency to adult humans (maybe some animals) - Material morality extends to speed bumps, overpasses, etc. - **Why this matters** - Would fundamentally change ethics as discipline - Moral responsibility would need reconceptualization - Can't just focus on human decisions anymore - Must analyze human-artifact assemblages - **Students should recognize the stakes** - This isn't just theoretical - If artifacts are moral agents, who's responsible when they "act badly"? - When Facebook's algorithm radicalizes someone, who's responsible? - The programmer? The company? The algorithm itself? - **Preview of chapter's response** - Will argue this goes too far - Artifacts influence moral behavior but aren't moral agents - Distinction is important for maintaining human responsibility > “ ‘We have been able to delegate to nonhumans ... values, duties, and ethics,’ he writes; ‘It is because of this morality that we, humans, behave so ethically…’ Echoing the views of Winner and Latour, the philosopher Peter-Paul Verbeek maintains, ‘The conclusion seems justified that artefacts have morality: technologies play an active role in moral action and decision-making.’ Verbeek refers to this phenomenon… as ‘material morality.’” --- ## Slide: "Objections to Artefact Agency" **Speaker's Notes:** - **Three main objections to consider** - **First objection: intention problem** - Moral agency requires intentional action - Acting with goal or purpose in mind - Artifacts don't have intentions - they're just caused to operate - Speed bump doesn't "intend" to slow cars - Therefore not moral agent - **Verbeek's response to intention objection** - Etymology of "intend": from Latin "intendere" = "to direct" - Artifacts clearly direct behavior - Highway overpass directs traffic patterns - Interface design directs user behavior - So artifacts do have intentionality in original sense - **Chapter's counter-response** - Verbeek is equivocating on "intend" - Directing one's own action ≠ directing others' actions - Intentionality requires psychological states: beliefs, desires - Artifact lacks these mental states - Just because it affects behavior doesn't mean it intends - **Second objection: freedom problem** - Moral agency requires freedom, non-deterministic action - Artifacts are purely determined by causal forces - Humans have "mysterious, non-deterministic agency" (Johnson) - This is necessary for moral responsibility - **Why this objection is weak** - Assumes libertarian free will exists - But this is highly controversial in philosophy - Many argue human action is determined too - Hard determinism: all events causally determined - Yet we still hold people morally responsible - So determinism doesn't rule out moral agency - **Third objection: responsibility problem** - This is the strongest objection - Moral agency traditionally entails moral responsibility - We praise and blame moral agents - But can't hold speed bump responsible for anything - What would it mean to blame an overpass? - **Why responsibility matters** - Moral responsibility links to punishment and reward - We imprison people for crimes, not artifacts - We give medals to heroes, not tools - If something can't be held responsible, how is it a moral agent? - **This sets up next slide's discussion** - Can we separate agency from responsibility? - Or are they necessarily linked? > “One obvious objection… is that although such artefacts have effects, they do not really ‘act,’ since they cannot form intentions… Since moral agency requires that the agent perform actions intentionally… artefacts… cannot be moral agents… [A]rgument… that humans possess a ‘mysterious, non-deterministic agency.’… Yet another argument… is that moral agency entails moral responsibility, but… it would make no sense to hold artefacts responsible for their ‘actions.’” --- ## Slide: "Separating Agency and Responsibility" **Speaker's Notes:** - **Floridi & Sanders's move: decouple concepts** - Moral agency ≠ moral responsibility - Can have agents who aren't responsible - This would allow artifacts to be agents without being responsible - **Evidence: dogs and children** - We say "bad dog!" when dog misbehaves - We praise children for good behavior - But don't hold them fully morally responsible - They lack capacity for full moral responsibility - Yet we treat them as proto-agents - **Application to artifacts** - Similarly, can say "these speed bumps are good" - "Those overpasses are evil" - Praising and blaming artifacts without full responsibility - This shows we already have concept of agency without responsibility - **This response seems to help material morality thesis** - Solves the responsibility problem - Artifacts can be moral agents in diminished sense - Like children: morally relevant but not fully responsible - **But chapter argues this reveals deeper problem** - If artifacts aren't responsible, who is? - Can't just be no one - Eliminating responsibility doesn't solve problem, it worsens it - **The accountability gap** - Call it "moral responsibility vacuum" - Bad outcomes occur - But can't locate responsible party - This is dangerous for justice and reform - **Guns example (Verbeek's)** - "Agency in assembly of gun and person shooting" - Distributed between human and artifact - But guns don't respond to moral reproaches - Can't punish gun, can't reform gun - So what do we do? - **Chapter's point: must identify human responsible party** - Gun manufacturer? Gun seller? Gun owner? Lawmaker? - These are the agents we need to focus on - Attributing agency to gun obscures this - Makes responsibility harder to locate, not easier - **The "making responsibility elusive" problem** - Technology corporations love distributed agency talk - "The algorithm did it" deflects responsibility - "We're just a platform" avoids accountability - Material morality thesis inadvertently aids this evasion - **Students should recognize contemporary relevance** - Facebook: "We don't create content, users do" - YouTube: "Algorithm just recommends what people watch" - Uber: "We're not employer, just connecting people" - Each case uses "distributed agency" to evade responsibility - **So even weak agency thesis is problematic** - Not just wrong but harmful - Obscures rather than clarifies ethical analysis - We need to focus on human decision-makers > “Floridi and Sanders… argue that the concept of moral agency can be coherently separated from the notion of moral responsibility… We often praise or blame… dogs and children… although we recognize that they are not morally responsible… If artefacts are not subject to moral responsibility, then… the upshot… must be to make moral responsibility for whatever effects those artefacts have more elusive… [W]ho is responsible for this? It is not the artefact… but must be some human being.” --- ## Slide: "Why Artefacts Aren't Moral Agents" **Speaker's Notes:** - **Chapter's athletic analogy is brilliant** - Artifacts clearly influence athletic performance - Better bicycle improves cycling speed measurably - Better running shoes improve times - Aerodynamic suits give competitive advantage - **Quantifiable improvements** - Can measure exactly how much bike design helps - Wind tunnel testing shows drag reduction - Time trials show performance gains - These contributions are real and significant - **By Latour's logic, bikes should be athletes** - Bikes "do athletic work" - Performance is "delegated" to equipment - Cyclist and bike are "assembly" that performs - So why not call bike an athlete? - **But this is obviously absurd** - Athlete is the human, not the equipment - We don't give medals to bicycles - Don't celebrate "greatest bike of all time" - "Athlete" category is reserved for humans - **Why we don't extend "athlete" to bikes** - Primary interest is human performance - Equipment shapes context but isn't what we care about - Human achievement is what matters - Bikes are important but not in same way - **Exact parallel with morality** - Artifacts shape moral context and influence behavior - This is real and important - But primary interest is human decision-making - Human moral agency is what matters - **Therefore legitimate to exclude artifacts** - Not arbitrary human prejudice - But reflects what we care about in ethics - Ethics is about human choices, character, intentions - Artifacts matter but in instrumental way - **The reification problem** - "Reify" = treat abstract thing as concrete - Material morality reifies technology - Makes it analogous to humans when it's not - This is category mistake - **What to do instead** - Analyze how artifacts influence human moral behavior - This is important and often overlooked - But keep focus on human responsibility - Don't diffuse responsibility across human-artifact networks - **Implications for designers** - Your designs will shape moral behavior - But you remain morally responsible agent - Can't say "the artifact did it" - Must own the choices embedded in designs > “No one would argue that we should extend the concept of ‘athlete’ to shoes and bikes… Although artefacts shape the context of athletic performance in multiple ways, we are primarily interested… in human performance itself. In the case of morality… our primary interest lies in human decision- making, not in the influence of artefacts on it, we have legitimate reason to exclude artefacts from the category of moral agency.” --- ## Slide: Discussion - Artifacts and Corporations **Speaker's Notes:** - **This objection is sophisticated and challenging** - Chapter dismisses material morality: can't hold artifacts responsible - But we already treat non-human entities as moral agents - Corporations are legal persons with moral and legal responsibilities - Can't imprison corporation, yet we hold them responsible - **Corporate personhood analogy** - Citizens United: corporations have free speech rights - Corporations can be sued, fined, held liable - We say "ExxonMobil knew about climate change and lied" - Attributing moral agency to non-human entity - **"The market" and "the algorithm" as agents** - Common speech: "The market punished that decision" - "The algorithm recommended extremist content" - We already distribute agency beyond individual humans - Why is artifact agency different? - **The anthropocentric prejudice charge** - Maybe we just can't accept non-human agency - Human exceptionalism: only humans matter - But this might be bias, not principled distinction - Animals have some moral status; why not artifacts? - **Points to raise in discussion** - Corporate personhood is legal fiction, not metaphysics - Created for practical purposes (liability, contracts) - Ultimately corporations are humans organized in certain ways - Can "pierce the veil" and hold individuals responsible - **Algorithm case is more challenging** - Machine learning systems make decisions humans can't understand - "Black box" problem: can't explain why algorithm decided X - If we can't understand it, can we hold it responsible? - Or must hold designers/deployers responsible? - **Winner and Latour might be right that...** - We underestimate artifact agency - Our ethical frameworks are too anthropocentric - Material arrangements deserve more attention - These are valid points - **But chapter's response still stands** - Acknowledging artifact influence ≠ calling them moral agents - Can study how artifacts shape behavior without attributing agency - Keeping focus on human responsibility is practical necessity - Otherwise responsibility disappears entirely - **Connect to designer responsibility theme** - If we accepted material morality fully - Designer might say "I just made the tool, tool is responsible" - This lets designers off the hook - Better to maintain: designer is responsible for foreseeable effects > “The problem with extending moral agency to artefacts… is that it makes it harder for us to focus our ethical analysis where it is ultimately needed: on human decisions and actions… It reifies technology into something analogous to human moral agents, when in fact it is not.” --- ## Slide: Discussion - Hard Determinism **Speaker's Notes:** - **This question strikes at foundations** - Chapter dismisses freedom objection: free will might be illusory - But then admits this threatens entire moral framework - If hard determinism is true, what's left of any moral responsibility? - **Hard determinism explained** - All events, including human decisions, are causally determined - Given prior states + laws of nature, everything follows necessarily - No "could have done otherwise" - Free will is illusion - **Why this seems to threaten morality** - Moral responsibility requires: could have chosen differently - "Ought implies can" - only responsible if alternative was possible - If determined, no alternatives were possible - Therefore no one is responsible for anything - **Applied to design ethics** - Designer determined by training, culture, economic pressures - "Couldn't have designed differently" - So not responsible for harmful designs - Entire framework collapses - **But many philosophers argue determinism compatible with responsibility** - Compatibilism: free will and determinism both true - "Free" means acting according to your desires/reasons - Not: uncaused by prior events - As long as action flows from your character, you're responsible - **Compatibilist design ethics** - Designer determined, but designs still reflect their values/choices - Difference between: designing what you value vs being coerced - Responsibility attaches to character and judgment - Can still praise/blame even if determined - **But hard incompatibilism is also defensible** - Maybe responsibility really does require libertarian free will - Without it, praise and blame are just manipulative tools - No one truly deserves punishment or reward - Ethics becomes pure consequentialism: just trying to shape future behavior - **What's left if hard incompatibilism true?** - Still have reasons to prefer some designs over others - Functional designs work better - Safe designs cause less harm - But no one is "to blame" for bad designs - Just need to change incentives and constraints - **This might actually be liberating view** - Less moralizing, more problem-solving - Focus on systems that produce good outcomes - Not on punishing "bad" designers - Design ethics becomes engineering problem - **Questions for discussion** - Does moral responsibility require libertarian free will? - Can we maintain design ethics under determinism? - If no one is responsible, what motivates ethical design? - Is consequentialist approach enough? - **Connect back to chapter** - Chapter doesn't resolve free will question - But argues: whatever we say about humans applies to artifacts - If humans aren't free, still different from artifacts - Difference is in having desires, values, capacity for reasoning > “Many have argued that the apparent freedom of will that humans possess is an illusion, and that human action is just as determined as what happens to artefacts and physical objects in general. Those who argue this way nonetheless hold that, in some sense, people are moral agents.” --- ## Slide: "The Designer's Ethical Burden" **Speaker's Notes:** - **Even without artifact agency, problem remains** - Artifacts shape behavior across generations - Effects are subtle and often unrecognized - This places enormous weight on designers - **Two types of questions designers face** - Empirical: "Will this device work as intended?" - Engineers trained for this - Testing, prototyping, simulation - Ethical: "Should there be a device that does this?" - No parallel training or methodology - **The ethical question is fundamentally different** - Not about facts but about values - Not about prediction but about justification - Requires normative reasoning, not empirical testing - This is philosophy, not engineering - **What equips designers for normative questions?** - Design school teaches: aesthetics, function, materials, business - Maybe one ethics class (often applied ethics) - No training in moral philosophy, political theory, social criticism - Yet expected to make decisions requiring exactly this expertise - **The brief defense: many decisions predetermined** - Designer often given specific brief - Hospital design example: many ethical aspects fixed - Choice of building hospital already made - Designer just optimizing within constraints - **But this defense is limited** - Even within brief, countless ethical choices - Default settings, accessibility features, privacy protections - Each choice has ethical weight - And society increasingly expects designers to push back on bad briefs - **Historical shift in designer responsibility** - Used to be: "I just execute the brief" - Now: "I have responsibility to question the brief" - Tech workers refusing military contracts - Designers declining fossil fuel clients - This implies designers should make ethical assessments - **But this creates paradox** - If designers should make ethical assessments - But lack ethical expertise - How can they fulfill this responsibility? - Next slides explore formalized approaches (like VSD) > “Certain artefacts may shape the way people live, and the lives they lead… This places tremendous weight on the Designer as an ethical decision-maker… The question confronting the Designer is not simply the empirical question – Will this device do what it is supposed to? – but rather the ethical question – Should there be a device that does this?” --- ## Slide: "Value Sensitive Design (VSD)" **Speaker's Notes:** - **VSD as response to designer burden** - Formalized methodology for incorporating values - Systematic rather than ad hoc - Three-phase approach - Aims to make ethics tractable for designers - **Phase 1: Conceptual analysis** - Identify which values are relevant - VSD specifies 12 human values: - Human welfare, ownership, privacy - Freedom from bias, universal usability, trust - Autonomy, informed consent, accountability - Calmness, identity, environmental sustainability - Consider direct stakeholders (users) - Consider indirect stakeholders (affected parties) - **Why having list is helpful** - Designer might not think of all relevant values - List prompts consideration - Ensures nothing major is overlooked - Makes implicit values explicit - **Phase 2: Empirical analysis** - Test how design actually affects values - Observe user interactions - Measure outcomes - This grounds ethics in reality, not speculation - **Why empirical phase matters** - Designers often wrong about effects - "Unintended consequences" are common - Testing reveals actual impacts - Can iterate based on findings - **Phase 3: Technical analysis** - Explore different implementations - Find ways to best realize values identified - This is where design expertise matters - Creative technical solutions to value conflicts - **Appeal of VSD approach** - Systematic: follows clear procedure - Interdisciplinary: combines philosophy and empirical testing - Actionable: leads to concrete design choices - Has been successfully applied in various contexts - **VSD in practice** - Used in healthcare system design - Information systems - Urban planning - AI development - Growing adoption in tech companies - **Why VSD seems promising** - Doesn't require designer to be philosopher - Provides structure and guidance - Makes ethics manageable - Next slide shows application > “Value Sensitive Design involves a three-part approach, consisting of conceptual, empirical and technical analyses… VSD identifies twelve human values that are relevant to Design… In the conceptual phase of VSD, the way in which the Design may impact upon these values, as they are held by the stakeholders affected, is considered.” --- ## Slide: "VSD Example: Missile Systems" **Speaker's Notes:** - **Missile guidance system example** - System allows mid-flight redirection - Can change target after missile launched - Military technology with obvious ethical dimensions - **Conceptual phase analysis** - Main relevant value: human welfare - Key question: "Can engineer ethically design weapons?" - This is not obvious - some say no, weapons inherently unethical - VSD approach: appeal to just war theory - **Just war doctrine background** - Ancient tradition in ethics (Augustine, Aquinas) - War can be justified under certain conditions - One key principle: discrimination - Must target military, not civilian targets - Codified in UN Charter and Geneva Conventions - **Application to missile system** - Re-directable missiles enable precision - Can avoid civilian casualties - "Intended to destroy military targets with pinpoint accuracy" - Therefore supports just war principle - Therefore supports human welfare - Therefore ethically permissible to design - **Design question that emerges** - What level of automation for redirection? - Fully manual: human decides whether to redirect - Fully automated: system decides based on algorithms - Hybrid: automated recommendations, human final decision - Each option has different value implications - **Empirical phase for this question** - Test prototypes with different automation levels - Observe human decision-making under pressure - Measure accuracy, speed, error rates - Find optimal balance - **Technical phase** - Implement chosen automation level - Interface design for human-machine interaction - Fail-safes and override mechanisms - This is where design creativity matters - **Why this example seems successful** - Clear value identification - Principled ethical reasoning - Practical design question - Empirical testing possible - Leads to concrete design choices - **But next slide shows problems lurking beneath surface** > “In the conceptual phase, human welfare is identified as the main relevant value, and the pertinent issue as ‘whether an engineer can ethically design a weapon’… Since re-directable missiles are ‘intended to destroy military targets with pinpoint accuracy and thus minimize civilian casualty loss,’ the Design in question supports the value of human welfare… The main Design issue arising… is the level of automation to be implemented in the redirection process.” --- ## Slide: "Problems with Value Sensitive Design" **Speaker's Notes:** - **Manders-Huits's critique reveals deep issues** - **Problem 1: Empirical interpretation** - Example: "67% of users fear security issues" - What does this mean? How do we use this data? - Do all 67% mean same thing by "security"? - Privacy-security? Financial-security? Physical-security? - How does 67% security fear relate to 49% privacy fear? - Empirical results are ambiguous, hard to operationalize - **Why this is serious problem** - Empirical phase is supposed to ground ethics in reality - But if results are uninterpretable, this fails - Designer still making judgments about what results mean - Just now with veneer of scientific objectivity - **Problem 2: The naturalistic fallacy** - Naturalistic fallacy: deriving "ought" from "is" - Can't get normative conclusions from descriptive facts - Knowing what values people hold ≠ knowing what values should guide design - VSD assumes: once we know stakeholder values, we know what to do - **Example of naturalistic fallacy** - Survey finds users prefer convenience over privacy - Does this mean designer should sacrifice privacy? - Users might not understand privacy risks - Might have preferences shaped by bad existing systems - "What people want" ≠ "what's ethically right" - **Problem 3: Conceptual phase requires ethical expertise** - Most fundamental problem - Conceptual phase identifies relevant values and principles - Missile example: appeals to just war theory - But doesn't argue for just war theory's correctness - **Why this matters** - Align with accepted theory ≠ align with true theory - What if just war theory is wrong? - What if there are better ethical frameworks? - Designer can't engage with these questions without philosophical training - **Value conflicts still unresolved** - Different ethical theories give different answers - Utilitarian: minimize total deaths - Deontological: never use people as mere means - Virtue ethics: what would courageous/just person do? - VSD doesn't resolve conflicts between frameworks - **Manders-Huits's conclusion** - "VSD requires complementary ethical theory and expertise" - Can't do ethics by methodology alone - Need actual philosophers involved - VSD is tool, not substitute for ethical reasoning - **Additional problem Albrechtslund raises** - Even if ethical theory is correct - Still problem linking theory to design - Missile example: precision weapons support just war doctrine - But: could also be used for surgical strikes on civilians - Terrorize population by threatening precision attacks - Design's ethical status depends on use, which is unpredictable - **So VSD doesn't solve the burden** - Helps structure thinking - But doesn't eliminate need for ethical expertise - Might give false confidence - "We followed VSD, so it's ethical" > “What does it mean if 67% of the users say that they fear security issues?… There is an implicit assumption in the methodology of VSD that one will know what to do in a normative sense, once these values are known. This, I contend, is where VSD runs the risk of committing the naturalistic fallacy… ‘VSD requires complimentary ethical theory and expertise’… [Technology that allows] more discriminatory attacks could just as easily be used to selectively destroy or threaten civilian targets.” --- ## Slide: "The Designer as Invisible Tyrant?" **Speaker's Notes:** - **The central question crystallizes** - "Who are Designers to make ethical decisions?" - Designers lack particular ethical expertise - Yet make decisions affecting millions - These decisions appear arbitrary to wider community - **The expertise problem** - We don't let untrained people practice medicine - Don't let non-lawyers argue in court - Don't let non-engineers design bridges - But we let designers make fundamental ethical choices? - **Designer's values might be sensible** - Not saying designers are bad people - Values probably as reasonable as anyone's - Might care about sustainability, accessibility, justice - But: no special training or legitimacy - **The legislation problem** - Designer isn't just making personal choices - Embedding values in material culture that shapes others' lives - This is "legislating" values for wider community - But designer has no democratic mandate - **Comparison to other value-imposing roles** - Legislators: democratically elected - Judges: appointed with oversight, legal training - Clergy: people voluntarily join religious communities - Teachers: licensed, curricula have public input - Designers: none of these legitimating factors - **The mediating effects magnify the problem** - Much of design's influence is subtle - Users don't recognize they're being shaped - Can't consent to what they don't perceive - This makes designer especially powerful - **"Shadowy figure" language is deliberate** - Designers work behind scenes - Most people don't think about design choices - Who decided phone should ding? Why this interface? - Invisible influence is most effective influence - **"Quietly foisting values upon unsuspecting public"** - Not malicious intent - But structural position - Every design choice privileges certain values, ways of living - Public often unaware this is happening - **Historical parallels with concern** - Urban planner Robert Moses shaped New York - Decisions affected millions, minimal democratic input - Technocratic vision imposed on population - This is cautionary tale for designers - **Contemporary examples** - Tech companies designing attention economy - Few hundred designers at Facebook, Instagram, TikTok - Shaping mental habits of billions - No democratic oversight or accountability - This is the "invisible tyrant" made visible - **So what to do?** - Next slides explore possible responses - None are fully satisfactory - Revealing deep tension in design as practice > “If the Designer has no particular ethical expertise… The Designer will, of course, have his or her own values… But the Designer seems no more entitled to ‘legislate’ those values than any other citizen. Yet, this is precisely what happens… The Designer, in this context, becomes a shadowy figure quietly foisting her values upon an unsuspecting public.” --- ## Slide: "Response 1: Prioritise User Agency" **Speaker's Notes:** - **First response: dodge the value-imposition problem** - Don't create systems that impose values - Instead create systems that preserve user choice - Present options rather than making decisions - This seems to avoid "invisible tyrant" problem - **Nordmann's philosophy of technology argument** - Technology's purpose: increase human control over nature - Reduce unknown background forces - Enhance human agency - This is the point of technology - **Nordmann's critique of nanotechnology** - Nanotech operates at imperceptible level - Humans can't perceive or control nano-processes - Therefore flawed technology - Reduces rather than enhances human agency - **Apply to Design generally** - Design should increase user agency, not decrease it - Don't impose ways of acting or perceiving - Give users control and choices - Let them make informed decisions - **How this answers "who are you to do ethics?"** - Designer doesn't do ethics - Just creates tools for users' ethical choices - Reins in mediating effects of design - Leaves room for informed consent - **Appeal of this approach** - Respects user autonomy - Seems politically neutral - Designer avoids paternalism - Democratic: users decide for themselves - **But chapter identifies serious problem** - Not desirable to avoid all value-imposing design - Maybe not even possible - Complex devices need defaults to be usable - **Photocopier example returns** - Photocopier with no default settings - User must set: paper size, orientation, sides, quality, etc. - Every single time - Would be so time-consuming as to be useless - Some defaults are necessary - **Smartphone example** - Imagine phone with no defaults - Must configure every setting before each use - Privacy settings, notification preferences, display options - Would take hours - Device becomes effectively unusable - **Further problems with pure agency approach** - Some value-imposing designs are good - Speed bumps reduce agency (must slow down) - But prevent child deaths - Seatbelt requirements reduce agency - But save thousands of lives annually - **The paternalism might be justified** - People make bad decisions with full information - Cognitive biases, present-bias, limited attention - Sometimes need defaults that protect from own mistakes - "Nudge" architecture serves this purpose - **So pure agency approach doesn't work** - Can't completely avoid imposing values - Shouldn't want to avoid all value-imposition - Need more nuanced approach - This motivates next response > “Nordmann… argues that the raison d'être of technology is to increase our control over nature and reduce the extent to which nature operates as a set of unknown background forces beyond our control… We might extend Nordmann's analysis to Design in general, arguing that Designers should not create systems that impose actions or ways of perceiving upon users, but only ones that present them with choices… [However] a photocopier with no default settings at all would be so time-consuming to use as to be effectively useless.” --- ## Slide: "Response 2: Participatory Design" **Speaker's Notes:** - **Second response: democratize the value decisions** - Don't have designer decide alone - Users and designers co-design - This seems inherently democratic - **Participatory design methodology** - Emerged in Scandinavia 1970s - Originally for workplace technology - Workers participate in designing tools they'll use - Has spread to other contexts - **Appeal of participatory design** - Designer doesn't impose personal values - Design reflects community values - Users have voice in what shapes their lives - Seems to solve legitimacy problem - **How this addresses invisible tyrant problem** - No longer shadowy figure making decisions - Transparent, inclusive process - Design emerges from community, not imposed on community - Democratic rather than technocratic - **Successful examples** - Community-based urban planning - Disability activists designing accessibility features - Open-source software development - These show participatory design can work - **But chapter identifies limitations** - **Limitation 1: Still risks naturalistic fallacy** - Adopting values people actually hold - Not engaging in moral analysis - "Community wants X" doesn't mean "X is ethically right" - Jim Crow laws were popular in communities that enacted them - **Limitation 2: Expertise problem re-emerges** - Stakeholders' ethical qualifications are dubious - Maybe less qualified than designer - Public might not understand technical implications - Might prioritize short-term wants over long-term needs - **Limitation 3: Which stakeholders?** - Direct users are obvious - But indirect stakeholders affected too - Future generations affected by environmental choices - How to include them? - Non-human animals affected by some designs - How to weight different stakeholder groups? - **Limitation 4: Whose values win when conflict?** - Stakeholders disagree among themselves - Privacy-concerned vs convenience-focused users - Majority rule? Protect minorities? Weight by intensity? - Participatory design doesn't resolve these conflicts - **Practical limitations** - Time-consuming and expensive - Works for community-specific designs - But mass-market products affect millions - Can't have millions participate meaningfully - **So participatory design helps but isn't complete solution** - Valuable in appropriate contexts - But doesn't eliminate need for ethical expertise - And has practical limits to scalability - **Alternative mentioned: separate ethicist role** - Qualified ethicists work with designers - Already done in healthcare protocols - Philosopher as team member - If problems are too much for design, relieve design of them - **This final alternative is intriguing** - Acknowledges ethical expertise is real - Divides labor appropriately - Designer does design, ethicist does ethics - But also admits design can't handle its ethical burden alone - **Implications of needing separate ethicists** - Design is not the humanist synthesis Modernism hoped for - Must be broken apart into specializations - Designer doesn't unify aesthetic, functional, ethical - Returns to fragmentation that design was supposed to overcome > “Participatory Design… the idea that Designers and users co-design, thereby creating an entity that is better adapted to its users… [It] suffers from many of the same limitations… it runs the risk of committing the naturalistic fallacy… [S]ee the suggestion… that Design teams include a ‘values advocate’… If the ethical problems raised by Design are too much for that practice to resolve, then perhaps ultimately it must be relieved of them.” --- ## Slide: Discussion - Precision-Guided Missiles **Speaker's Notes:** - **This question goes to heart of VSD example** - VSD concluded: precision missiles serve human welfare - Minimize civilian casualties compared to alternatives - Therefore ethically permissible to design - **But does "less bad" = "good"?** - Precision weapons kill fewer civilians than carpet bombing - But still kill people - Does relative improvement make it ethical? - Or is the whole project unethical? - **The designer's complicity question** - Weapon wouldn't exist without your design - You are causally necessary for its use - Every death it causes traces back to your work - Are you responsible for those deaths? - **Designer might say: "I design, others use"** - My job is just technical design - Political leaders decide whether to use weapons - I'm not responsible for use, only for making it work - This is the "guns don't kill people" defense - **But this seems like evasion** - You know how weapon will be used - Use is predictable, not accidental - "I just followed orders" was rejected at Nuremberg - Why should "I just designed it" be accepted? - **The "humanitarian weapons designer" contradiction** - Can you design instruments of death humanely? - Is this like being a humane executioner? - Some say: if war is inevitable, better weapons minimize suffering - Others say: participating in killing is always wrong - **Different ethical frameworks give different answers** - Pacifist: Never design weapons, full stop - Just war theorist: Design weapons that enable discrimination - Utilitarian: Design if total suffering reduced - Deontologist: Might say using people as mere means always wrong - **Further complications** - Precision weapons might enable more wars - Political leaders more willing to use force if "clean" - Drone strikes have no political cost (no US casualties) - So "humanitarian" weapons might increase total violence - **The designer's knowledge problem** - Can't predict all consequences - Precision missiles might be used for evil purposes - Might be sold to tyrants - Might malfunction and kill civilians anyway - Designer can't control these factors - **Questions for discussion** - Should designers accept military contracts? - Is there moral difference between weapons and other defense technology? - What about defensive vs offensive weapons? - Does working for "our side" matter ethically? - Should engineers refuse weapons work en masse? - **Historical examples to consider** - Manhattan Project scientists: many later regretted work - Agent Orange developers: claimed not to know uses - Land mine designers: International ban passed - Each case raises question of designer responsibility > “In the conceptual phase, human welfare is identified as the main relevant value, and the pertinent issue as ‘whether an engineer can ethically design a weapon.’… [R]e-directable missiles are ‘intended to destroy military targets with pinpoint accuracy and thus minimize civilian casualty loss.’” --- ## Slide: Discussion - Google and "Don't Be Evil" **Speaker's Notes:** - **This scenario is barely fictionalized** - Google's motto was literally "Don't be evil" - Business model is surveillance capitalism - Massive data collection enables targeted advertising - This is not conspiracy theory; it's public business model - **The designer's dilemma** - You work as lead designer at Google - Want to "not be evil" - But company requires surveillance for revenue - How do you resolve this? - **VSD empirical testing reveals** - Users accept surveillance for free services - Majority prefer free-with-tracking over paid-without - Privacy concerns exist but are outweighed by convenience - This is real data from real user studies - **Participatory design says** - "Give users what they want" - Users want free services - Accept data collection as trade-off - Therefore surveillance is ethically acceptable - **The question: Does chapter give grounds to refuse?** - VSD found relevant values - Empirically tested user preferences - Participatory approach would defer to users - Each methodology leads to: build the surveillance system - **Why this is such a good question** - Reveals how methodologies can ratify existing power structures - "Users want it" might just mean "users are used to it" - Preferences shaped by available options - No genuinely privacy-respecting alternatives exist - So users "choose" surveillance - **The manipulation problem** - Users don't fully understand implications - Terms of service are unreadable - Data uses are opaque - "Informed consent" is fiction - So empirical testing doesn't capture genuine preferences - **Broader issues with revealed preferences** - Economist method: look at actual choices - But choices constrained by options available - And shaped by information asymmetries - Not clear they reflect "true" values - **Does chapter's framework help?** - Aristotle's account: is surveillance necessary for good to come to be? - Communication and information access are genuine goods - But does achieving them require surveillance? - No - could have paid model, public funding, other options - So surveillance serves business model, not genuine needs - **What designer should do** - Refuse to design surveillance features? - Try to change business model from within? - Resign and blow whistle? - Work on privacy-preserving alternatives? - **Real-world cases** - Tristan Harris: ex-Google ethicist, now critic - Facebook employees protested political manipulation - Some tech workers refuse surveillance projects - But: individual action doesn't change system - **Structural problem** - Designer's ethics can't fix corporate incentives - Market rewards surveillance (it's profitable) - Regulation is needed but slow - Individual designer is trapped - **Questions for discussion** - Is staying and trying to reform from within effective? - Or does it just provide ethical cover? - Should designers organize collectively to refuse certain projects? - What responsibility do users have to reject these services? - Is this ultimately political problem beyond design ethics? - **This reveals chapter's limitations** - Focuses on individual designer's choices - But many problems are structural, systemic - Can't be solved by better individual ethics - Need changes to economic incentives, regulation, power structures - Design ethics might not be enough > “There is an implicit assumption in the methodology of VSD that one will know what to do in a normative sense, once these values are known. This, I contend, is where VSD runs the risk of committing the naturalistic fallacy.”