Adversarial epistemology (in politics)

I watched Obama's speech last Friday afternoon. As usual, I really enjoyed it -- it's nice to remember what it was like to have a president, and I like the rambling / wonky style he can indulge now that he doesn't have to be so on-message. Link.

He suggested that some Adversary wants people to believe that they can't affect politics enough to make efforts worthwhile; that's good for the Adversary because it's already in power and this lets it keep power. I hadn't thought of this, and it's a problem for me, because I don't think I can affect politics enough to make efforts beyond informed voting / possibly donation worthwhile. (I think Obama was mostly telling people "Adversary wants to deceive you into thinking voting is not worthwhile", but once you take the Adversarial hypothesis, it's not clear that it should be limited to voting.)

This is a game-theoretic problem I haven't seen much analysis of: when an Adversary can partially manipulate your perception of the world, how do you adjust your epistemics to make a good decision? This looks like an intermediate case between Descartes' Demon and a plane dropping leaflets that say "Surrender!" -- I know more about the Adversary than Descartes did, but less than a soldier who's seen the leaflets drop.

A few scattered thoughts on how to deal with this situation:

Cui bono? To identify a hidden actor from some actions / outcomes, you can ask "who benefits?" and guess that the beneficiary may be the hidden actor. In the adversarial epistemics case, my actions are partially determined by the Adversary, so I can look at the actions and beliefs of mine and ask "who benefits?" (Obama is partially using cui bono to support the adversarial hypothesis in the first place. "Don't think voting will work? Who benefits from that?")

Attack model: We can try to model the Adversary's capabilities and motives, and use this to figure out what our defense options are. For example, how specifically can the Adversary target me? Presumably they are playing against some classes that I'm part of, not directly against me, and presumably they are spending more effort attacking classes that will benefit them more per dollar. Maybe this gives me a way to trust some info to be "cleaner" than other info?

Epistemics vs morale: I'd naively guess that it's easier to attack morale than to attack epistemics. If there's an Adversary, I think there's more reason to trust spreadsheet-style calculative models based on historical data ("I should help in way X because its track record / expected value is good") over gut-level models ("I should help because it feels intuitively likely to work").

Distraction: It's plausibly easier to discourage people by directing their attention toward an intractable issue than it is to make a tractable issue look less tractable; e.g., I think I'm particularly powerless over Supreme Court confirmations, so directing my attention to confirmations makes me feel more powerless. This suggests I should look away from the spotlight for better opportunities.

Based on this, I might reconsider donation to political causes; not sure which ones, and they'd probably be justified in terms of Citizenship instead of in terms of Effective Altruism.

Realistically, I doubt I have the time / energy for political action beyond voting :(

The Late Sleeper: a problem for ataraxians

Dan just wrote about Benetar's "Better Never to have Been," and this neat thought experiment occurred to me when I was talking to Killian about it.

ETA: here's what I should have written. Some people think that so-called "good experiences" are actually just the cessation of suffering, reprieve from craving, etc. That would mean that the best you can do is break even; that's good, but at best it's only as good as it would have been to not have had those cravings / sufferings in the first places. I'm calling this the ataraxian position; see also antifrustrationism and tranquilism.

Does this fit with our experience? Well, it seems like sleeping people are free from suffering/craving, so sleep is an optimal state (along with nonexistence or complete cessation of suffering) under these theories, and it's less confusing than nonexistence. Now let's compare supposedly "positive" experiences to being asleep. If you think "positive" experiences can't be better than being asleep, you're an ataraxian. If you think "positive" experiences can be better than being asleep, you're not an ataraxian. I'm not an ataraxian.

(Overly complicated original post follows)

This doesn't address Benetar's case directly, but it deals with a related view that I've heard: the view that the "best" outcomes/experiences for people are actually just a minimization of their suffering. For example, eating a tasty meal just amounts to temporarily bringing your suffering from a lack of tasty food down to zero; it's good, but at best it's only as good as it would have been never to have been hungry at all. If you think that the best anyone can do is hit the neutral point (instead of having "positive" experiences), then it's better never to have been born. I'm calling this ataraxian ethics -- I'm sure it has some real name, but I don't know what it is.

Here's the thought experiment that makes me reject ataraxian ethics. In case it gets famous, I'm obligated to give it a name and a little story:

The Late Sleeper
Alice is sleeping on a Saturday morning. When she sleeps, she is truly unconscious, and is not experiencing any kind of suffering. There's some chance that she'll happen to wake up early, and if she does, she'll see a beautiful sunrise and enjoy a cup of coffee with a loved one. Whether she wakes and sees the sunrise or sleeps and doesn't see it won't have any further effects on her life. Would it be good for her to happen to wake up early?

The ataraxian would say that this would not be good; when Alice is asleep, she isn't suffering, and according to the ataraxian it's not possible for her to be in a better state than this. At best, this experience could be as good as remaining asleep, but in order for this to be the case, Alice-at-sunrise would have to be in her literally best possible state, completely free of suffering. If there's any chance of a bad experience during the sunrise (e.g. Alice's coffee is a little too bitter), it would be better for her to remain asleep. In fact, there's no experience at all (!) Alice could have, no matter how "good", that would be better than staying asleep.

This makes me pretty sure I'm not an ataraxian; there are a million things that sound better than being asleep, and I'd be willing to take some suffering (e.g. being sleepy later or risking injury by walking down the stairs) for many of them. I'm pretty sure I'm not just being culturally pressured into saying this, and if I'm fooling myself, I think I've fooled myself into actually enjoying those experiences.

I'm tempted to say that people who might think they're ataraxians should obviously agree with this and stop being ataraxians (probably retreating to saying "OK, there are actual good things beyond cessation of suffering, but the bad outweighs the good in basically every life, so it'd still be better not to exist"), but who knows, people think lots of different things :)

You can sing along with "The Cross of Coronado"

This will probably be my only contribution to film music studies! I searched around, and didn't find anyone else who'd pointed it out.

This is the "Cross of Coronado" theme from Indiana Jones and the Last Crusade, and it plays every time the cross shows up onscreen:



In addition to being a killer motif, the rhythm sounds like the phrase "The Cross of Coronado"! Listen again and try singing along; you'll never be able to hear the theme again without hearing the words in your head.

Whose values should advanced AI have?

When we're talking about getting advanced AI to behave according to "human values", some people respond by saying something like "which human values?", "whose values?", or "there's no such thing as 'human values', don't you know about [cultural diversity / relativism]."

I sometimes worry that people raising this question aren't being sincere -- instead, they're trying to paint people worried about AI safety as socially and politically naive techno-futurists ("nerds", which we don't like any more), and they're happy to ask the question, show they're superior, and then leave without actually engaging with the question.

Putting this aside, I think "what exactly should we do with powerful AI, and how should we decide?" is a natural political question. However, I also think it's not too different from most other political questions.

For some advanced AI systems, we should probably should treat them like most other artifacts -- they will be used to pursue the goals of their owners, and so the answer to "which values" for "Private Property AI" will be "some of the values of their owners, or some values that their owners want them to have instrumentally to achieve their owners' ends". Use of Private Property AI should be subject to some laws, which might include what values and capabilities Private Property AI is allowed to have, and we'd be hoping that these laws in combination with economic forces would lead to advanced AI having a positive overall impact on human welfare. (It seems like markets have done OK at this for other technologies.)

The private ownership solution unsatisfying if we think that some kinds of advanced AI distributed through markets is likely to be bad for overall human welfare (the way we think that distributing nuclear or biological weapons through markets would be bad for overall human welfare). If advanced AI is powerful enough, it could create super-huge inequality between owners and non-owners, allow owners to defy laws and regulations that are supposed to protect non-owners, or allow owners to control or overcome governments. Owners might also use advanced AI in a way that exposes humanity to unacceptable risks, or governments with advanced AI might use it to dominate their citizens or attack other countries.

In response to this, we'll probably at least restrict ownership of some advanced AI systems. If we want more powerful AI systems to exist, they should be "Public AI", and have their goals chosen with some kind of public interest in mind.

At this point, it seems like the conversation has usually gone to one of two places:

1. We need to solve all of moral philosophy, and put that in the AI.
2. We need to solve meta-ethics, so that the AI system can solve moral philosophy itself.

This leads to questions like "Should advanced AI be built to follow total hedonic utilitarianism? Christian values? Of which sect, and interpreted how? Which ice cream flavor is best? What makes a happy marriage? How can we possibly figure all of this out in time?"

I don't think it's true that we actually need to solve moral philosophy to figure out what values to give Public AI. Instead, we could do what we do with laws: agree collectively to give Public AI systems a tiny set of agreed-upon values (e.g. freedoms and rights, physical safety, meta-laws about how the laws are to be updated, etc.), leaving most value questions like "which ice cream flavor is best" or "what God wants" to civil society.

Political theory / political philosophy have spent some time thinking about the same basic question: "we have this powerful thing [government], and we disagree about a lot of values -- what do we do?" Some example concepts that seem like they could be ported over:

• Neutrality / pluralism except where necessary: don't have governments make decisions about most value-related questions; instead, just have them make decisions about the few they're really needed for (e.g. people can't murder or steal), and have them remain basically neutral about other things.
• Enumerated powers: "The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people." Contrast with letting governments do whatever isn't explicitly prohibited.
• Rule of law: instead of giving governors full power, make a set of laws that everyone including governors is subject to.
• Separation of powers: make it hard for any set of governors to take full power.
• Constitutionalism: explicit codification of things like the above, along with explicit rules about how they can be updated.
• Democracy: give everyone a voice in the creation and enforcement of law, so that the law can evolve over time in a way that reflects the people.
• "Common-value laws": I'm not sure if there's a real term for this, but a lot of laws codify values that are widely shared, e.g. that people shouldn't be able to kill each other or take each other's stuff at will.
• "Value-independent laws": again not sure if there's a real term, but some laws aren't inherently value-related, but are instead meant to make sure that civil processes that generate value for people (like trade) go smoothly.

I think "constitutional democracy" is the right basic way to think about the "whose values" problem for Public AI, and makes the whole thing look at lot less scary.

Media about good people, plus masculine heroes

Not everything has to be Breaking Bad, Game of Thrones, True Detective, House of Cards, etc. -- "morally ambiguous" (i.e. bad) characters fighting each other or trying to beat the world, or characters who don't really try to help each other creating drama for basically no reason (Gray's Anatomy except for the parts where they're doing surgery). Here are some things where characters are trying to be good, but there are hard challenges for them to overcome, and they try their best to team up and succeed:

• The Force Awakens
• Stranger Things (esp. season 1)
• The Good Place (h/t Nicole for pointing this out; the Parks and Rec mafia seems to have figured out how to write comedy with actual good people in it, e.g. Brooklyn Nine Nine)
• Most Ghibli movies
• Shin Godzilla (translated as Godzilla: Resurgence)
• The Martian
• Harry Potter, during the parts where the characters aren't making Teen Drama
• The Lord of the Rings

(Most of these fit a genre Killian came up with, "adventure," which maybe I'll write about at some point. Think Stargate or Indiana Jones, not The Bourne Identity. I don't think this is a coincidence; adventures give these kinds of characters a place to live, since they don't create their own drama by screwing things / each other up.)

This is a difference between Classic Star Trek (Next Gen, TOS, Voyager) and Discovery, and a difference between The Force Awakens and Rogue One (though the characters in Rogue One sometimes say the right words, they don't seem to mean them, and the director doesn't seem to believe in Good People Trying to Help Each Other).

Good People Trying to Help Each Other doesn't require characters to actually be perfectly good, considerate, heroic, etc. -- I think Hopper in Stranger Things 2 is a good example of a character who clearly has weaknesses, but is trying to deal with them in a sensible way (by just apologizing in a vulnerable way to the people he's hurt).

A related point is that some people try to remember Han Solo as some kind of "morally ambiguous" (i.e. bad) mercenary, but watching the original trilogy he's clearly a good, sensitive person wearing mercenary clothes. All of the male leads (Luke, Han, Obi-Wan, Yoda, to some extent Vader) in the original Star Wars trilogy are doing a masculine hero thing that's very different from what I remember from most masculine heroes of the last 20 years; they feel a lot of different emotions and express them openly (not just anger, but also excitement, panic, happiness, sadness, etc.), they're very focused on their relationships with other characters (not just Avengers quips or Firefly will-they-won't-they friendships/relationships), and they have to try really hard to succeed (not James Bond).

(I actually think this is the primary problem with the Prequel Trilogy, and the reason that The Phantom Menace feels the most like Star Wars -- after Qui-Gon dies, there are no good people left, and so no Star Wars happens for the rest of the movies; it's just Obi-Wan and Anakin doing a bad job of acting "cool".)

Rewatching, it's surprising that Han Solo is remembered as "cool" -- it's a really different version of "cool" than I see other places. Props to folks like Dwayne Johnson and Channing Tatum for attempting to bring this back in some of their work, and to The Force Awakens for building it into all of their characters -- nobody in that movie is "cool" at all, not even the bad guys.

(I'm not sure how these things do with feminine characters, and I suspect I can't really connect with those characters the way a more feminine person could. I'd love to know what more feminine people think about feminine heroes in these or other media.)

I think these kinds of media are fun, energizing, and (suspicious moral claim) ennobling to watch -- they're the kind of media where we can tell ourselves stories about how we should act, how we should treat one another, and how we should respond to tough situations. They believe that some outcomes and behaviors are good and others are bad, and that we can actually try to help each other do good things instead of acting as "cool" as possible in a meaningless world. Ultimately, that's why I think this kind of media is important; True Detective is beautiful, but it just can't be used to communicate who we want to be.

Metaballs in ~100 lines!

ThreeJS is pretty neat!

Three cards for a Mill draft deck