Response post #2

It's been a while, so I wanted to reply to at least one comment:

Post: Observer selection effects from scratch
Comment: here

I’m not sure it makes sense to assume that nothing about your observation X favors any of these observers over any other (out of the 101 possible observers distributed among W1 and W2)...

For instance, let’s say... observation B is externally focused (i.e. that apples appear to fall downward). For observation B, it seems that some relevant factors would be how available you think the information is to make that observation (i.e. are there apple trees near others), how likely it is that other people are paying attention, etc. If it seems reasonable that others could be observers too, it seems like that should tilt you more towards believing that you are in W2. I’m not sure, but it seems plausible to me that you should think it’s more than 100 times more likely that you’re in W2 than W1.

Thanks for this! I'm not sure that I understand it, though. As far as I can tell, you're saying that some facts about observation B and world W2 could tell us about how likely each observer in W2 is to observe B, right? Like if almost all of the observers in W2 are near apple trees and B is that apples fall downward, it's likely that many observers in W2 will observe B, but if all of W2's observers are in the middle of the ocean, probably few of them will observe B. Is that right?

If so, I totally agree, and I think it's saying the same thing as my post. In my post, I'm not talking about all observers in W1 and W2, but only about those that do actually observe X. In your example, that'd mean there is 1 observer in W1 who sees an apple fall, and 100 in W2 who see the apple fall. That would be in part because of how many observers total there are in each world, but also largely because of where those observers are placed, whether they're paying attention, etc, and it'd be consistent with W1 having millions of observers far from apple trees, asleep, etc -- W2 would still be favored, even with millions of sleeping observers.

Let me know if I've made a mistake, but I think we're on the same page?

Liveblog: attempting to break a work block

"Work blocks" are, as far as I can tell, my biggest current work problem right now. The basic scenario is:

1. There's a project that's the right thing for me to do now. (Important, time-sensitive, etc.)
2. I don't actually work on it.
3. I don't work on anything else either.

I have a work block today. My time log looks like this:

Task	Start	End	Duration
Email	7:30	8:10	40 min
[other]	8:11	8:12	1 min
[other]	8:12	8:25	13 min
[other]	8:25	8:28	3 min
Important Task	8:55	9:04	33 min lost, 9 min worked
[other]	9:29	9:47	25 min lost, 18 min worked
Block-breaking	11:58		2 hours 11 minutes lost

The warning signs are pretty clear; I spent a little time on a few things (email is fine, but the others look not-great to me), stalled a half hour before really starting on the Important Task, worked a little on it, switched to something else after incurring a long delay, then stalled for another 2 hours (!). When I look at my past work logs, this is pretty typical, and adds up to a day where I only get a few hours of real work done despite a lot of time in my office.

I can't quickly tell whether I usually get the important task done on the day that I attempt it (i.e. whether those few hours actually accomplish what I wanted to do), or whether it usually has to wait until later in the week. I am pretty confident that I get these tasks done in the week that I attempt them, though.

An obvious option is to work on other things. I don't like this option much; Important Tasks tend to hang over me, and they usually are really time-sensitive.

My experience of these things is that I don't feel like doing them. Lately, they've been pretty complicated pieces of work. They're not something I can do without really paying attention; typically it feels like there's not an easy entry point. In this case, I have a list of parts of the project, and it doesn't feel easy to start on any one of them without loading the whole thing into my brain. I don't feel like loading it into my brain, so I can't start.

[Goes to look at list of parts] A lot of these are also kind of vague, like "figure out x".

[Looks at list again] What if I just had to do one of these? That seems like it wouldn't be hard. I think they feel like they interfere with one another. Maybe I'll just pretend like they don't. That seems like a thing to try.

...after I take a shower.

[Later] Now I'm giving this a shot. Starting on the first subtask...
[Later] Actually trying now.
[Later] Augh, actually doing it now.

Oh gosh, this is not fun, but it is happening. Each of these is a little puzzle, and would be easier to deal with if I had some information that I don't have. First subtask finished, moving to the next.

This seems to have worked, though I'm still slower than I'd like. I'll keep using this when I encounter projects like this that have subtasks; maybe I'll get better at it.

Update: this process seems to have worked well this time, though it worked a lot better when I started fresh on it this morning. I do tend to be better at getting going on tough tasks from a standstill than when I've already worked some, so I'll probably try to identify similar tasks in the future and wait to try them until I can start first thing in the morning.

Also, live-blogging has been much more successful than my previous attempts to solve this problem. +1 for thinking out loud, I guess!

Bostrom's observation equation

Nick Bostrom's book "Anthropic Bias" seems to be the most thorough examination of observation selection effects around. However, I don't really understand the reasoning method he proposes at the end of the book (Chapter 10) and in this paper. So, let's work through it.

(Note: I started this post not understanding the observation equation, but now I feel like I do. +1 for writing to understand!)

First, Bostrom states the Strong Self-Sampling Assumption (SSSA), an informal rule for reasoning that he thinks is correct:

(SSSA) Each observer-moment should reason as if it were randomly selected from the set of all observer-moments in its reference class.

Sounds pretty good to me, but the devil's in the details -- in particular, what is a reference class?

Bostrom offers an "observation equation" formalizing SSSA. Suppose an observer-moment \(m\) has evidence \(e\) and is considering hypothesis \(h\). Bostrom proposes this rule for \(m\)'s belief:
\[P(h|e) = \frac{1}{\gamma}\sum_{o\in O_h\cap O_e}{\frac{P(w_o)}{|O_o\cap O(w_o)|}}\]
Okay, what does this mean? Ignore \(\gamma\) for now; it's a normalizing constant that depends only on \(e\) that makes sure probabilities add up to 1, I think. \(O_h\) is the set of observer-moments that are consistent with hypothesis \(h\), and \(O_e\) is the set of observer-moments that have evidence \(e\). So, what we're doing is looking at each observer-moment \(o\) with evidence \(e\) where hypothesis \(h\) is actually true, and adding up the probabilities of the worlds that those \(o\) live in, divided by the number of observers in that world that are members of \(o\)'s "reference class", which we still haven't defined.

Now let's look at the normalization constant:
\[\gamma = \sum_{o\in O_e}{\frac{P(w_o)}{|O_o \cap O(w_o)|}}\]
This is pretty similar to the above, but it just sums over observer-moments that have evidence \(e\). In fact, the inside of the sum is the same function of \(o\) as the inside of the sum of the first equation. In fact, I think we can sensibly pull this out into its own function, which semantically I think is something like the prior probability of "being" each observer:
\[P(o) = \frac{P(w_o)}{|O_o\cap O(w_o)|}\]
For each observer, the prior probability of "being" that observer is the probability of being in that world, split equally among all observers in that world that are in the same "reference class". This in turn lets us rewrite the observation equation as:
\[P(h|e) = \frac{\sum\limits_{o\in O_h\cap O_e}{P(o)}} {\sum\limits_{o\in O_e}{P(o)}}\]
This is useful, because it makes it clear that this is basically the formula for conditional probability!
\[P(h|e) = \frac{P(\text{observe }e\text{ and }h\text{ is true})}{P(\text{observe }e)}\]
So, now I feel like I understand how Bostrom's observation equation works. I expect that I'll mostly be arguing in the future about whether \(P(o)\) is defined correctly, and I still need to come back to what exactly an observer's "reference class" is. Spoiler: Bostrom doesn't pin down reference classes precisely, and he thinks there are a variety of choices.

Observer selection effects from scratch

Suppose that I have only three theories T0, T1, T2, describing three possible worlds W0, W1, and W2. Now, suppose that I observe X, and suppose that the following is true:

• In W0, there are no observers of X.
• In W1, there is one observer of X.
• In W2, there are 100 observers of X.

What should I now believe about my theories? Should my beliefs be sensitive to how many observers of X there are in each world?

It seems pretty clear to me that I shouldn't believe T0, since it's not compatible with my observation of X; that's a minimal level at which my beliefs should be sensitive to the number of observers of X. A way of justifying this is to cash out "I believe in Tn" to mean "I believe I am in Wn", or "I believe that my future observations will be consistent with Tn". Then "I observe X" and "In W0, there are no observers of X" come together to imply "It's not possible that I'm in W0" and hence "I don't believe T0".

What should I think about T1 and T2, though? It's still possible that I'm in either one of their worlds, so I'll believe both of them to some extent. Should I believe one of them more than the other? (Let's assume that T1 and T2 were equally plausible to me before this whole thing started.)

Pretty solid ground so far; now things get shaky.

Let's think about the 101 possible observers distributed among W1 and W2. I think it's meaningful to ask which of those I believe I am; after all, which one I am could imply differences in my future observations.

Nothing about my observation X favors any of these observers over any other, so I don't see how I can believe I'm more likely to be one of them than another one, i.e. I should have equal credence that I'm any one of those observers.

This implies that I should think it's 100 times more likely that I'm in W2 than in W1, since 100 equally likely observers-of-X live in W2 and only one observer-of-X lives in W1. I should think T2 is much more likely than T1. This answers the original question of this blog post.

However, that means that if I'm considering two cosmological theories, and one of them predicts that there are billions of copies of me having the experience I'm having now, I should believe that it's very likely that that theory is true (all else equal). It's weird that I can have that kind of belief about a scientific theory while I'm just sitting in my armchair. (Nick Bostrom calls this "The Presumptuous Philosopher", and thinks you shouldn't reason this way.)

So, it seems like we have to pick one of these weird things:

1. It's nonsensical to have beliefs about which possible observer I am (even if being different observers implies different future observations).
2. Something besides my observations and my prior belief in theories of the world should affect my beliefs (in theories of the world, or in which observer I am).
3. Just by sitting in my armchair and thinking, I can come to strong, justified beliefs about cosmological theories based solely on how many people-thinking-in-armchairs they contain.
4. I've made some other mistake in my reasoning; like, my account of theories and worlds is wrong, or I'm not thinking carefully enough about what it means to be an observer, or I'm not thinking clearly about normative principles around beliefs, or something else. (Actually, me making a mistake wouldn't be so weird.)

?!

I tend to lean toward 3 (well, if I assume 4 isn't true), but smart people disagree with me, and it's kind of a crazy thing to believe. It could also mean that we're Boltzmann brains, thought I'm not sure. See also this paper.

---

Addendum: consider this similarly plausible-sounding reasoning:

1. "I observe X" just means "there exists an observer of X".
2. "There exists an observer of X" rules out T0, but not T1 or T2.
3. "There exists an observer of X" doesn't favor T1 or T2.
4. All else equal, I should have equal belief in T1 and T2.

I think this reasoning is too weak, and leaves out some implications. "I observe X" implies "there exists an observer of X", but I'd argue that it implies some additional things: it has implications about what I should believe I'll observe in the future (not just what some existing observer will observe), what theories I should believe are true (not just some observer), and which observers I should believe I could possibly be (ditto). Maybe I should redo my earlier reasoning in terms of expected observations and see what happens? 

Response post #1

I've got a couple of great commenters, so I wanted to do a quick response post to encourage them to keep commenting (If they'd like to!). Aspirationally titled "Response post #1" -- we'll see if there's a #2.

Post: "Animal Rights"
Comment: here

I think I agree with all the premises of this article. And I think I agree on your categorization of rights as pragmatic. I don't think I agree that we should give animals rights. Buuut, I might have misunderstood, and I might be too stuck in my ways.

First, my understanding: if I'm reading it right, "animal rights" means, among other things, legally enforced veganism. Like, if we can't have animals as property, we probably definitely can't kill them for food, and it's hard to imagine how we could get a cow's agreement to give us milk. (correct me if I'm wrong.)

 Yeah, I agree that animal rights almost certainly mean we can't have them as property. I'm undecided on whether this actually means that you couldn't have an arrangement with a cow where you could get milk from them; we get labor from humans, after all, so there might be a legal arrangement where a cow could be "paid" (in nice living conditions, luxuries, etc?) to give milk. Obviously it's super-hard to do this, and might not be feasible (or oversight might be too expensive), but it's not obviously impossible to me.

Imagine that there was a service that could be performed only by people who weren't able to express themselves linguistically or understand language. I can imagine that an appointed guardian might be able to set up an employment situation that would work for everyone. However, this would be a lot more expensive than just, you know, owning the cows!

Breeding is another huge challenge -- it just doesn't seem likely to be acceptable.

(btw I realize "legally enforced veganism" would totally derail this conversation on many blogs; I'm hoping yours doesn't have the kind of readership that would do that)

Yeah :) Also, jumping suddenly to legally enforced veganism (or something very close to it) wouldn't work -- one of the big policy challenges would be avoiding a Prohibition-like reaction where production is pushed underground and the policy is reversed later anyway.

This seems a bit extreme if we actually just want to improve animal welfare. You can have laws around how you treat property, while it's still property. (I think?) Like, I can buy a car but I can't just drive it around wherever, or let my kid drive it, or ghost ride it. You've got to be responsible with cars (where "responsible" means a basket of things that not everyone agrees on); why not just work on defining the basket of responsibilities with animals?

Why not go all the way and do this with people as well? Welfare is what I really care about, so why not be libertarian and allow people to own other people (maybe only if it's mutually agreed upon), and define a basket of responsibilities for people-owners?

My answer, I think, is something like this:

• "defining a basket of responsibilities" is a building-up process -- we have to manually add pretty much everything we care about.
• "rights" are a way of dramatically limiting acceptable behaviors in one shot, and with lots of room for interpretation or refinement later by judges. We sort of know what a "right to freedom" means, or a "right to dignity", and we can later judge whether those things are violated through legal interpretation.

It seems to me like rights are more appropriate for people. It's very lucrative to own a person, and (unlike cars) the space of things you should or shouldn't be able to do with a person is very big, messy, and ill-defined. Also, we've societally decided that it's better to err on the side of restricting what people can do with people more rather than not enough. I think these arguments apply well to animals, as well.

The argument that "we can't legally protect animals effectively while they are property" is an old animal rights argument (see e.g. this, which refers to attempts to protect slaves' welfare without giving them the right to not be property), and I haven't done my homework on the debate over it, but it seems like a plausible argument to me.

On to your proposal of animal rights: How do you "represent" an animal in a decision? How can you represent a critter who can't ever communicate with? I guess you could get, say, a cow expert to tell you about all the things that make a cow's life better or worse. But again, we're in vegan world, so there probably wouldn't even be cow experts anymore. And we're far enough away from our current world that my instincts are probably not great moral judgment tools anymore. So, maybe I'm just too stuck in my ways.

I think this is a great point, and it might turn out that this is a thing we can't figure out how to do. I'm guessing there'll still be animal experts or cow experts, but that's totally a guess. If it turns out we can't get milk, my official answer is ¯\_(ツ)_/¯

---

Post: Peak Experiences
Comment: here

Re. your concern about being too navel-gazey/introspective: it didn't feel that way at all for me, I really liked this post! I have a similar concern as a lot of what I write is very introspective, but I generally just try to make it clear I'm not suggesting my experiences necessarily generalise to others - just that they may be sufficiently similar to be interesting to others. Or I try to frame it more as "here's an experience of mine that made me think about a more general problem/thing, here are my thoughts on the more general thing." Regardless, this post made me think about my own peak experiences in a really useful way (and in a way I hadn't done before), so I don't think you're too much at risk of being overly navel-gazey yet! :) 

Thanks! That's reassuring, and a good suggestion. I think I'll replace my self-deprecating "me" tag with an "examined life" tag to counteract my bashfulness about introspection :)

No blog today

Sorry guys :(

Animal rights?

I was intrigued by this article; I think it correctly points out that caring about animal welfare is pretty different from caring about animal rights, but I was hoping for more of an argument in favor of animals having rights. So, I decided to think about this a bit myself.

At first, I thought that I wouldn't tend to be in favor of animal rights. I generally think about ethics in terms of welfare instead of in terms of rights or obligations, so why would I think animals should have rights? However, after thinking about it more, I've come down in favor of animal rights, and I feel like I have a better understanding of why human rights matter.

So, let's talk about human rights. I haven't seen a metaphysical argument that humans "naturally" have rights, and I'm not sure I'd be convinced by that kind of argument anyway. However, there area couple of reasons that I think it makes sense to assign humans rights:

• Humans have a strong preference for self-determination (which is partially final and partially instrumental)
• Rule-consequentialist / rule-utilitarian: "human rights" are a good policy to agree on, because that policy helps us maximize welfare

"Should we decide to give animals rights?" is the natural question for me; we could decide on rights pragmatically, and then follow them dogmatically, even when we don't see why they're useful in a particular case. I generally don't think the first reason above applies to animals, but I think the second does. So, I think we should give animals rights.

(A note on why I think it makes sense to consider rights as pragmatic things we decide on: rights are pretty complicated, sometimes seem inconsistent or made-up, and they're constantly up for debate. For example, what's the deal with children's rights? What are all the trade-offs and edge cases around the right to free speech, or the right to refuse service? I think it used to be considered a "right" of soldiers to defend themselves and their fellows on a battlefield, and to be exempt of moral blame when they follow orders, but revisionist just-war theorists are working on overturning that, IIRC. These things are clearly cultural constructs, and we should choose them as we see fit.)

Some rights that I think probably make sense for animals:

• They should be represented in decisions (e.g. political decisions)
• Guardianship should be a matter of political debate (and animals should have representatives in these decisions)
• If rights that work for kids don't work for animals, then animals should have more rights than kids do (e.g. people care about kids intrinsically, usually, and don't benefit financially from having them, whereas animals don't have these natural protections)

On this last point: imagine a world where having children could be very lucrative. In this world, we'd probably have to restrict the right to have children, and give children rights that prevent parents from exploiting them for financial gain. I think we probably will need to extend those kinds of rights to animals.

So, animal rights: yes! I'm just not sure which ones, or how we get there.

No blogs this weekend

Not blogging this weekend because I have lots of socializing to do, and my next few topics are ones I want to spend enough time on to do well! See you monday.

What I'd like to see studied

Sometimes I daydream about having an organization in charge of making sure the world goes well. There are some things that I'd want them to be studying (not an exhaustive list, since I'm tilted toward listing weird / counterintuitive things):

Cosmologically big questions:

• Boltzmann brains
• Many-worlds interpretation of quantum mechanics

Pressing moral questions:

• How exactly are animals morally relevant?

Practical questions:

• We keep having kids and adding them to the world. What are their lives going to be like? We're in charge of their lives and educations for around the first 18 years of their lives. How do we set them up for good lives, and what do good lives look like? What do we tell them, and what cultural goals do we set for them? (Maybe I'll post about this.)
• How do communities, cities, and countries work? What are their goals, and how are their policies doing at achieving those goals?

Beyond studying these questions, I'd also want this organization to have a Global Status Monitor. It'd give statistics on the whole world, so that you can take in at a glance what the state of play is; ideally, it'd also have the ability to scrub back through history to see how things have changed. (The monitor doesn't need to be instantaneous; I'd be happy if it updated once a year, for instance.)

• What is everyone on Earth doing for work?
• How does everyone's labor fit together?
• What do they eat, how do they find shelter?
• What are people's overall financial pictures?
• How is everyone's social life going?
• How is everyone's mental and physical health?
• What political things are happening?
• What social change / civil rights things are happening?
• What are the cutting edges of science?
• What are the cutting edges of art?

Boltzmann brains

What if almost everything we thought we knew about our position in the universe was wrong? What if we actually were not members of a species that arose around 200,000 years ago, among life forms that started evolving around 4 billion years ago on a planet that formed 4.5 billion years ago, in a universe that began with a Big Bang around 13 billion years ago? A single cosmological discovery that changed all of that would be an amazingly big deal (at least in terms of scientific knowledge -- it might not change what we actually do with our lives).

That's roughly what's at stake with the question of Boltzmann brains -- whether instead of the picture above, it's more likely that we came into existence a short time ago via random (quantum or thermal) fluctuations during an extremely long quiet period in one of the last ages of the universe. Not only our ideas about our position in the universe are at stake; it's also possible that only my brain arose this way, perhaps a few minutes or seconds ago, meaning that much of what I think is real (other people, places beyond my immediate reach, all of human history, etc.) is not actually physically real.

Now, this sounds suspiciously similar to many radically skeptical arguments, like the brain in a vat thought experiment -- how do you know you're not just a brain in a vat? These arguments are great for an intro-to-Philosophy class, but once the shine wears off, they seem a little thin -- what does it really offer to say "well, you might be a brain in a vat, there might be a deceiving demon, etc.", and what more can we say about these arguments? They might be useful thought experiments for epistemologists who need corner-cases to test their ideas of what "knowledge" really is and what we can really know, but they don't feel productive as a way to think about the world. I think the typical arc is to be surprised by these arguments, live with them for a while, and then forget about them, and I think that's fine.

However, I think the Boltzmann brains (BB) argument is importantly different. The BB argument isn't "how do you know you're not a BB", it's "according to some cosmological theories, many BBs will exist, and using some kinds of anthropic reasoning, it's likely that you're one of those BBs." It's as if scientists pointed their telescopes at the sky and saw vast arrays of brains-in-vats; we have (as far as I know) real reasons to take the BB scenario seriously.

I haven't been able to find a comprehensive survey of argumentation around BBs, or even a very rigorous paper that attempts to thoroughly examine the question; it's usually treated as an example or interesting implication in cosmology or philosophy books and papers, as far as I can tell. It looks like it's only been seriously considered for about 20 years, like so many of the ideas that I think are most important.

To be totally honest, I expect the BB argument to fail. I also don't think it's likely to be importantly action-informing; how would I really make decisions differently if I were a BB? However, it's one of a few really big questions about what the world is actually like that I'd really love to see answered. In fact, I think I'll post again to talk more about those big questions -- stay tuned.

I'm playing with the idea of writing and thinking more about BB -- it's an appealing hobby project. If I do, you'll see it here first!

Writing about not writing

Well, I stayed up late hoping that I'd think of something to blog about, and no dice. This was probably a bad call. My current criteria, which may be too strict:

• Not work: I started blogging at all because I found myself only talking about work, and was afraid I was getting boring.
• Don't use willpower: nothing that I feel like I'm forcing myself to write.
• Not always introspective: doing too much of this leaves a bad taste in my mouth.

Strategies for coming up with posts in the future:

• Write about the first thing I can think of: in this case, I had another introspection topic that I could have written on, but I didn't feel like doing another introspective post right this minute.
• Write about work: I'd prefer to avoid this, but whatever I'd blog about would be different from my day-to-day work, and that might be nice.
• Write about something I thought about a while ago: I have a few technical topics lying around that I could convert into blog posts quickly. The main blocker is that that doesn't sound very fun; if I've already done the thinking, then I'm just left with the hard part of expressing it.
• Write about my past: for example, I used to be pretty into Buddhism; I could write about what that was like and what changed. Many other memoirish things could work.
• Journal: I could write about things I've done recently, or about how I think life is going. Not super-appealing, but maybe I could figure out what's fun about it. (I was really bad at journaling when I was a kid, and I'm probably not great now, never having done it.)
• Media reviews: books, movies, TV, etc.
• Micro-posts: collect a few topics where I only have a few sentences to say.

Activities that might help generate ideas I'm actually excited about:

• Watch TV or read a book.
• Read a technical paper or go down the rabbit hole on some topic of interest.
• Take a walk? (Feels un-promising, but based on experience this could work)
• Talk to K about something, then write it down.

Particular topics that might work for tomorrow, though they don't sound that appealing to me right now:

Work:

• Intelligence explosion
• A concrete setting for MIRI's Löb problem
• Different threads in AI safety
• EA and x-risk / AI safety

Philosophical / technical / "intellectual":

• Boltzmann brains
• Natural selection or abiogenesis in Conway's game of life
• Patient-centric ethics
• Some philosophy of mind prompt from Luke

Other:

• My recent vacation
• Consequentialist vs expressive modes of being
• Segments of a Dungeons and Dragons campaign
• Music I've liked recently

Peak Experiences

Lately, I've been using the phrase "peak experiences" to refer to really unusually good experiences. Maslow defined peak experiences as "rare, exciting, oceanic, deeply moving, exhilarating, elevating experiences that generate an advanced form of perceiving reality, and are even mystic and magical in their effect upon the experimenter", and associated them with "self-actualizing individuals". I'm not sure I'm describing the same thing, and I'm skeptical that the experiences I'm talking about have to do with "self-actualization". For now, just beware that I'm using "peak experience" to mean "really really good experience".

From talking to friends, I get the impression that some people have fairly infrequent peak experiences that are much better than their average experiences, and some people don't. I'm the former sort of person; operationally, my peak experiences are good enough that I think I'd be willing to trade between a day and two weeks of average experiences for a single peak experience (lasting maybe 30 seconds to a few minutes), assuming all other effects of this trade are neutral. I'm not sure what my rate actually is because I haven't figured out how to make that kind of trade, especially in a way that makes other effects of these trades neutral.

How frequent are my peak experiences, and what are they like? I think I need a more explicit picture of what peak experiences look like in order to get a feeling for frequency; if I don't really know what they look like, it's hard for me to retrospectively say how often they happen (though I could just keep a counter in the future to get better data). I also think it'd be nice to have better vocabulary for peak experiences because then maybe I'd notice and appreciate them more, the same way having a better vocabulary for experiences of food, music, or movies helps me appreciate them more.

Here's an attempt at listing categories of peak experience that I've had, with made-up names that are supposed to get the gist across. I've left out a couple of things that aren't polite to blog about, and are probably obvious :) I'm sure I'm missing some categories or cutting things the wrong way in some cases, and it'd be interesting to find out whether other folks share these categories.

Frisson: actual frisson, a physical tingle or shiver along with a feeling of intense emotional response and emotion to an idea, a piece of music, or both. Often includes tearing up. It's a little embarrassing to give examples of what causes this in me, and I'm not sure why; it feels closely linked to the deepest and scariest-to-expose parts of my personal experience.
- Duration: up to a minute?
- Frequency: several times a week, probably?

On Fire: a feeling of excitement and cockiness about my work and the work of my community more generally. A feeling that my particular skills and idiosyncrasies line up really well with what I'm doing with my life. Sometimes related to reflecting on recent achievements. Kind of manic, but in a pleasant way. Check out how this guy dances; confident! (The end of that video is great, by the way.)
- Duration: a few minutes to 15 minutes?
- Frequency: once every week or two?

Deep Connection: hard to describe; a feeling of the removal of barriers between me and a person or people I'm with, or between me and the rest of the world. It feels like a lot of machinery that normally goes into charting my individual course through the world is turned off, and I'm not calculating risks and benefits anymore. I haven't had this happen without alcohol and a very comfortable social setting (that I'm in or have just left). If I'm alone, music helps a lot.
- Duration: a minute?
- Frequency: not sure; no more than a few times per year.

Pinch Me: this is a feeling I get when I realize I'm in a situation that's way outside what I expected in my gut (in a good way). For example, standing in the back of a water taxi speeding between islands in Venice at sunset (humblebrag!). It's sort of like a really nice form of incredulity, a feeling that a long-shot bet taken on a whim has paid off, or a feeling that I don't have any reasonable right to be experiencing this, but not in a way that pulls me out of the moment or makes me feel guilty about it.
- Duration: a few minutes to a half-hour?
- Frequency: maybe once or twice a year. By their nature, it seems like these can't happen very often.

⁂

A few notes from making this list:

• Three of these categories are basically solitary, and the fourth (deep connection) I think I experience in a basically solitary way, despite it being about removing barriers between me and others / the world. This feels like a property of my intense experiences generally (good and bad). Is this correct, or typical?
• Music is a big factor!
• Higher-order awareness: it's tempting to think that these feelings are naturally higher-order ones, where being aware of the experience is part of the experience. I'm not sure this is right; certainly the only peak experiences I can report are ones that I took note of (e.g. by noticing the distinctive physical sensations of frisson), but I think these experiences would still be good if I wasn't taking note of them. Maybe there are whole categories that I wasn't very aware of, and so can't report!
• Flow is missing; I don't think I've experienced it. Ditto for anything involving raising kids.

This post felt pretty navel-gazey, and I'm generally skeptical that introspection of this kind translates well to writing -- everyone's experiences are different, and the insights that I have about myself might not be very applicable to others. I think I'll try to limit posts that are just about me and my experiences, but this is one I've been meaning to write for a while, so I'm glad I got it out there. (It also resulted in me changing my list of categories, which is useful for me.)

Like a Boss

(This is not the blog post I meant to write, but I kinda got into it! Fun!)

Eventually, I might want or need to be a boss.

I don't think this is a natural role for me, and I think in general it's fairly hard to do well. I've been impressed with my current boss (good thing this blog is anonymous), so I wanted to take some notes. If I ever do become a boss, I'll probably do a couple interviews with good bosses to find out more.

So, here's my current simplistic model of how to be a boss (in fields like mine, where people aren't necessarily filling pre-defined roles):

0. Hire people you feel really good about; a good rule of thumb might be being confident that they'll be great at at least one mission-relevant thing.

1. Ask the person to do something that is mission-relevant and that they're good at. You probably hired them for their ability at some task, or with a suspicion that they'd be good at something; start them there. If you didn't hire them, maybe ask the person who did. If you strike out here, move to step 2.

My most likely mistakes on this step:

• Trying to get this person to do whatever is most mission-critical, regardless of whether they're good at it.
• Not asking them to do anything because what they're good at doesn't seem mission-critical enough.
• Not wanting to give people tasks because I think I'd be better at them. (This is a mistake because you have to push through this in order to get gains from having employees.)
• Asking a person what they want to do; this just pushes the boss' job onto them.

2. Figure out what (else) they're good at, among mission-relevant things, and ask them to do those things. Most people are good at a variety of things, and some of those will be mission-relevant. Ways to do this:

• Ask them what they think they were best at at their previous job.
• If they had a good boss or good co-workers at their last job, ask those people what they were best at.
• Guess what they might be good at, and ask them "How would you feel about doing x? I'm wondering if you'd be good at general class of things X, and this would be a good way to get some information about it."
• Ask them if they've seen anyone else at your org doing a kind of task they think they'd be good at. This is a little risky, since it kind of pushes the boss' job onto them.

My most likely mistakes on this step: again, probably being too exacting about what is mission-relevant.

There are some other things that seem good, but less critical, to me:

• Gauge autonomy; different people need different kinds and amounts of guidance.
• Give performance feedback.
• Actually care about their life, and show it.
• Get data about how their work is going, and demonstrate that you won't abuse that data (and that you'll use it to help them). I suspect "what an employee is doing" is very mysterious, and any data you can get will help, but giving data (like a timesheet of what they did and how long it took) is a vulnerable place for an employee, so you have to acknowledge their fears of being judged and demonstrate that you won't abuse this privilege.

Overall, the thing that a boss seems to need to do, in addition to being cool to people, is hold a picture in their head of what tasks are mission-relevant and what capabilities employees have, and then pair people with tasks in a sensible way. I'm sure there's a lot more to it, but I didn't have this basic model before, so it feels like progress to me.

Hair-trigger mood and imposter syndrome

I can definitely relate to this post. I have a somewhat different take; maybe this means that I'm experiencing a different psychological situation with similar symptoms, or maybe it's a different view on the same basic phenomenon.

When things start to go wrong -- either mundane things like persistent messes or less-than-optimal personal interactions, or bigger things like an entire day where I don't get my work goals done -- I tend to feel much worse than I think the situation warrants. How to explain this?

I think it feels like my brain is in danger mode -- like my brain thinks my life is on the edge of total collapse, and one thing going poorly could push things over the edge. This could be in terms of my own happiness (that my personal life satisfaction could be pushed from positive to negative because of one mishap), or in terms of success at my various projects (that my work or relationships could fall apart because of one mistake). This is strange, because I get a lot of evidence regularly that things are going fine, and positive feedback from bosses, co-workers, and loved ones that I'm doing well.

More specifically, it feels to me like I'm on the edge of failing to fulfill some roles, like the role of Good Boyfriend, Good Employee, or Competent Adult. This focus on roles, along with the positive feedback I'm obviously getting, suggests to me that what's actually going on is something like imposter syndrome.

Imposter syndrome fits: I get a lot of positive feedback because I can keep people fooled, but just a small number of slips could let someone see through the façade. I didn't think about imposter syndrome before, because I usually associate it with doubting abilities, and I'm pretty confident in my abilities; it's my ability to fit these roles, or maybe something like my virtues (diligence, conscientiousness, general competence at being adult) that are at play here.

So what do you do about imposter syndrome? A few obvious options:

1. Get better calibrated, and come to appreciate that I'm actually fulfilling these roles pretty well (my performance is higher than I thought).
2. Decide not to try to play these roles.
3. Find out that I have the wrong idea about these roles and what they imply (the requirements are lower than I thought; the thing I'm an imposter of is imagined).

I think I've actually had the most success with option 3. The main thing that's happened lately is that I've found out that hardly anyone fulfills these roles the way I have in mind -- nobody (or almost nobody) is the Competent Adult or Good Employee that I'm thinking of, and I'm actually well within the typical distribution of performance. This has just come from talking to people more openly about my difficulties and hearing that they have similar difficulties. My high standards were mostly imaginary.

Another thing that's helped is finding out that when people see my problems, they don't conclude that I'm an imposter. A big example is when I recently had a really bad work week. I logged like 10 hours of actual work, and dreaded having my weekly check-in with my boss. Instead of firing or (more likely) lecturing me, he said "eh, that happens to me sometime -- I'm a pretty high-variance worker", and asked whether anything was going on in my personal or work life that was affecting my ability to get work done. That was awesome!

I do worry that getting over this will lower my standards and result in worse performance, but I actually think it's more likely to improve my performance -- I waste enough energy worrying about this stuff, and being frozen / de-motivated as a result, that I think avoiding this kind of situation will more than offset any drop in performance from exorcising these imagined roles (and I haven't actually seen that kind of effect in practice at all yet, so I'm not sure it'll really happen).

Final note: this expansion of the applicability of imposter syndrome also provides a nice explanation for my difficulties with vacationing successfully -- I'm worried enough about being a Good Vacationer that I'm not paying attention to what I'd like to do in the moment. This suggests that option 2 -- abandoning the idea of playing roles -- might be the best solution here. I'm excited to give it a try!

Preludes and Haiku

Over the weekend I saw an awesome performance of Chopin's Preludes. The full piece is about 45 minutes, and it's made up of 24 short pieces. The shortest one is only 12 bars long! I really loved listening to these pieces, and I think the short (episodic?) structure contributed to this -- each one is like a taste of some particular thing, kind of like a multi-course meal.

I gather that a lot has been written about the Preludes from a musical point of view, but the thing that caught my eye was that preludes before Chopin's were almost always literal preludes to longer pieces, introductions to fugues or improvised (?!) introductions to other compositions. (When did improvisation stop happening during classical music performances?) Chopin's Preludes, on the other hand, weren't written to introduce anything, or were just introductions to one another. Chopin's Preludes apparently elevated preludes' status to an art form in and of themselves.

This reminded me of haiku, which I learned in the excellent book Bashō and his Interpreters used to be just the opening stanza of renga, a form of long collaborative poem created at poetry gatherings, before being elevated to stand-alone pieces. No great moral; I wonder if there are other opportunities to split off prelude-like things into forms of their own? Maybe movie trailers?

Incidentally, that Bashō book is pretty great! The author, Makoto Ueda, gives a selections of poems from throughout Bashō's life, interspersed with biographical context about what Bashō was up to (including maps of his travels!). Each poem is given in its original Japanese, a word-for-word English translation, and Ueda's personal translation to idiomatic English; Ueda gives notes explaining the context of the poem's composition and any references you need to understand the poem, e.g. to Japanese plays or traditions or puns; then Ueda lists some commentary from poetry critics throughout history. I'd love to read more poetry books that do this, since otherwise I'm not in a position to really understand the poems. I think it's probably the best overall introduction to haiku I've seen, and I definitely composed some after reading it.

Acts, omissions, friends, and enemies

The act-omission distinction plays a role in some ethical theories. It doesn't seem relevant to me, because I'm much more concerned with the things that happen to people than with whether a particular actor's behavior meets some criteria. (Of course, if some consequence is more beneficial or harmful if it is caused by an act or an omission, I'd care about that.)

Supererogatory acts, and the broader concept that some acts are required / forbidden while others are not, also play a role in lots of ethical theories, but don't seem relevant to me. To me, ethics is entirely about figuring out which acts or consequences are better or worse, and this doesn't give an obvious opening for making some acts required or forbidden.

I recently had an idea about why these concepts appear in some ethical theories: acts/omissions and supererogatory acts seem useful for identifying allies and enemies. This is roughly because acts tend to be costly (in terms of attention and other resources), and supererogatory acts tend to be expensive as well. There's not a lot more to say:

• Allies will pay the cost to help you through their acts; supererogatory acts are especially good indicators of allies.
• Enemies will pay the cost to hurt you through their acts.
• Neutral parties may hurt or help you through omissions, but since these aren't costly, they don't carry much information about whether that party is an ally or enemy; they don't seem to be thinking about you much.

From my perspective, this is a tentative debunking of these concepts' role in ethics, since allies and enemies don't belong in ethics as far as I can tell. For others, allies and enemies might be important ethical concepts, and maybe this could help them explain these concepts' role in those ethical theories.

Final note: I remember hearing about supererogatory acts' evil twins, i.e. acts that are not forbidden, but are morally blameworthy; "suberogatory acts" (search for the term on this page). These might be useful for identifying allies, who will avoid suberogatory acts, but they don't seem to play much of a role in any ethical theory.

Do reinforcement learning systems feel pain or pleasure?

Do reinforcement learning systems have valenced subjective experiences -- do they feel pain or pleasure? If so, I'd think they mattered morally.

Let's assume for the time being that they can have subjective experiences at all, that there's something it's like to be them. Maybe I'll come back to that question at some point. For now, I want to present a few ideas that could bear on whether RL systems have valenced experiences. The first is an argument that Tomasik points out in his paper, but that I don't think he gives enough weight to:

Pleasure and pain aren't strongly expectation-relative; learning is not necessary for valenced experience

An argument I see frequently in favor of RL systems having positive or negative experiences relies on an analogy with animal brains: animal brains seem to learn to predict whether a situation is going to be good or bad, and dopamine bursts (known to have something to do with how much a human likes or wants something subjectively) transmit prediction errors around the brain. For example, given an unexpected treat, dopamine bursts cause an animal's brain to update its predictions about when it'll get those treats in the future. Brian Tomasik touches on this argument in his paper. This might lead us to think that (1) noticing errors in predicted reward and transmitting them to the brain via dopamine might indicate valenced experience in humans, and by analogy (2) this same kind of learning might indicate valenced experience in machines.

However, I think there is a serious issue with this reward-prediction-learning story. This is that in humans, how painful or pleasurable an experience is is only loosely related to how painful or pleasurable it was expected to be. If I expect a pain or a pleasure, it might reduce my experience of pain or pleasure, but it doesn't seem to me that my valenced experience is closely tied to my prediction error; a fully predicted valence doesn't go away, and in some cases anticipating pain or pleasure might intensify it.

Biologically, it shouldn't be too surprising if updating our predictions of rewards isn't tightly linked to actually liking an experience. There seems to be a difference between "liking" and "wanting" an experience, and in some extreme cases liking and wanting can come apart altogether. Predicting rewards seems very likely to be closely tied to wanting that thing (because the predictions are used to steer us toward the thing), but seem less likely to be tied closely to liking it. It seems quite possible to enjoy something completely expected, and not learn anything new in the process.

In a nutshell, I'm saying something like:

In humans, the size of error between actual drive satisfaction and predicted drive satisfaction doesn't seem strongly linked to valenced experience. Valenced experience seems more strongly linked to actual drive satisfaction.

This seems to me like evidence against RL systems having valenced experience by virtue of predicting rewards and updating based on errors between predicted and actual rewards. Since this is the main thing that RL systems are doing, maybe they don't have valenced experiences.

If valenced experience doesn't consist of noticing differences between expected and actual rewards and updating on those differences to improve future predictions, what might it consist of? It still seems very linked to something like reward, but not linked to the use of reward in updating predictions. Maybe it's related to the production of reward signals (i.e. figuring out which biological drives aren't well-satisfied and incorporating those into a reward signal; salt tastes better when you're short on it, etc.), or maybe to some other use of rewards. One strong contender is reward's role in attention, and the relationship between attention and valenced experience.

The unnoticed stomachache

Consider the following situation (based on an example told to me by Luke about twisting an ankle but not noticing right away):

A person has a stomachache for one day -- their stomach and the nerves running from their stomach to their brain are in a state normally associated with reported discomfort. However, this person doesn't ever notice that they have a stomachache, and doesn't notice any side-effects of this discomfort (e.g. lower overall mood).

Should we say that this person has had an uncomfortable or negative experience? Does the stomachache matter morally?

My intuitions here are mixed. On the one hand, if the person never notices, then I'm inclined to say that they weren't harmed, that they didn't have a bad experience, and that it doesn't matter morally -- it's as if they were anesthetized, or distracted from a pain in order to reduce it. If I had the choice of giving a Tums to one person who did notice their stomachache or to a large number of people who didn't, I would choose the person who did notice their stomachache.

On the other hand, I'm not totally sure, and enough elements of discomfort are present that I'd be nervous about policies that resulted in a lot of these kinds of stomachaches -- maybe there is a sense in which part of the person's brain and body are having bad experiences, and maybe that matters morally, even though the attending/reporting part of the person never had those experiences. Imagine a human and a dog; the dog is in pain, but the human doesn't notice this. Maybe part of our brain is like the dog, and the attentive part of our brain is like the human, so that part of the brain is suffering even though the rest of the brain doesn't notice. This seems a little far-fetched to me, but not totally implausible.

If the unnoticed stomachache is not a valenced experience, then I'd want to look more at the relationship between reward and attention in RL systems. If not, then I'd want to look at other processes that produce or consume reward signals and see which ones seem to track valenced experience in humans.

Either way, I think the basic argument for RL systems having valenced experience doesn't work very well; none of their uses of reward signals "look like" pleasure or pain to me.

Costume design: Anakin Redeemed

@dfwaffles I want your alternate costume design for: Darth Vader survives ROTJ and needs good guy armor

— Chris Dewey (@CDooey) July 29, 2016

Don't mind if I do! A few basic problems:

• How to deal with the limbs? Vader has maybe half of his left arm left, and that's it.
• How to deal with breathing?
• Jedi robes, or what?

Thinking about this, I kept coming back to one key question: how would post-redemption Anakin think of himself? The appearance of Anakin's Force ghost implies that he was suddenly turned into a high-level Jedi Knight, but I find that a little implausible.

I think Anakin's recovery might go in three stages:

1. Very damaged

No limbs, maybe gets around in a power chair. At this stage, Anakin is resistant to the idea of ever getting robot limbs again, since he wants to be as far from Vader as he can be. He still needs a breathing mask all the time. He's going to need a caretaker -- probably C-3PO and a couple Alliance personnel.

2. Recovering

Anakin is now feeling secure enough to get robot limbs again, but not the Luke-V human-looking ones -- he doesn't want to forget that Darth Vader is still in him, and seeing the robot limbs helps with that. I don't think he's even comfortable being a Jedi yet -- I'd guess that he's wearing basic Tatooine clothing at this stage. He still needs a breather mask sometimes (like Han and Chewie's from when they're on the asteroid in V), but it turns out that Force healing plus bacta have mostly repaired his lungs and other health problems -- the Emperor was keeping him sick and dependent on his armor on purpose. He's decidedly not military, but reluctantly advises Luke on Force-related stuff, especially Dark Side things.

3. Jedi Master Anakin

Anakin now wears Jedi robes again -- I think he'd take after Qui-Gon, having gotten in touch with the living Force. He also has Luke-V human-looking limbs now -- he's embracing being a human, but the machine is always under the surface.

I'm most excited to tell stories about the middle one -- not totally down-and-out, but not back to being a Jedi again. Here's how I think he'd look:




Definitely no helmet; "Let me look on you with my own eyes." He only needs the breathing mask sometimes, when his concentration is broken, but he carries it everywhere. The breathing mask unit could be mounted in the middle of his chest as a nod to Vader's chestplate.

As long as I'm retconning, Anakin is now played by James Earl Jones, as it always should have been.

Another thought on this premise: Darth Vader is like a huge war criminal, right?! Luke can probably protect him, but they might keep him secret -- maybe claim that Anakin (Luke's dad) was a prisoner of the Empire, or something.

Killian's conspiracy theory is that Vader actually did live through RoTJ, and Luke just burned his armor and claimed he was dead to keep him secret. I asked about the Force ghost, and she said something about astral projection :)

Anything but this

Blog Brag, with Numbers

I had Many Achievements today that I want to brag about:

• 9 hours of work, only 1 of which was a meeting (I don't know how people put in 8 hours of real, actual work in a day. Do most people really do this? I guess most people in service industries and skilled/unskilled labor do. I'm amazed by this.) (I know I should count achievement of goals instead of hours spent -- hours are on the "cost" side of the balance sheet -- but in my case hours and achievement scale together, more or less.)
• 3 meals, including plants in each one (oatmeal and blueberries; salad (greens, carrot, and bell pepper) and a quesadilla; salad (same) and PBJ; ice cream)
• ~32 emails sent
• ~7 dishes and ~7 silverware washed
• 17 is the level I reached in Hearthstone :)
• 1 call to Grandparents

It was a really unusually good day, according to my metrics. Less time with K than I would have liked, so we'll have to see how I can fix that.

Brilliance, blunders, and backfires

In AlphaGo's recent play, there were two kinds of moves that stood out:

• Brilliant moves: moves that accomplish the system's goal – winning the game – in ways that a human wouldn't think of, and that might take a while to even be understandable in retrospect (or might elude our understanding altogether).
• Blunders: moves that humans can identify (though sometimes only in retrospect) as bad for the system's goal.

As AI systems become more capable, it will be harder to tell the difference between brilliant moves and blunders until their effects are felt, and even in retrospect they may be hard to diagnose. If hard-to-understand AI systems are given safety-critical or high-impact tasks, blunders could become a source of significant harm.

However, I think we should be at least as concerned about a third kind of behavior:

• Backfires: moves that accomplish the system's nominal goal, but that don't do what the user actually wanted or that have unintended side-effects, and that might only be identified as backfires in retrospect.

Like blunders, we have the challenge that backfires won't easily be distinguished from brilliant moves. Backfires bring additional challenges; unlike blunders, improving a system's ability to achieve its nominal goals won't fix backfires, and may actually make them worse:

• A backfire might accomplish what we really wanted, but with additional effects that we don't want – getting a ball into a hoop while also smashing every vase in the room, or making a cup of coffee while also lighting the house on fire or breaking the law. As systems become more capable, they will be able to cause broader effects, making this problem worse.
• A backfire might accomplish the nominal goal without accomplishing what we really want, e.g. by manipulating a reward signal directly instead of by winning games of Go or Atari. As systems become more capable, they will find more ways of accomplishing their nominal goals, making this problem worse.

Backfires could happen because it's difficult to specify in full what we want a system to accomplish and what unintended consequences we want it to avoid, difficult to know in advance what means a system might use to accomplish its nominal goal, and difficult to specify goals in a way that can't be "gamed".

A "new" Magic: the Gathering format

The rules:

• Put all your non-land cards in a big pile. (You can include lands with cool abilities in this pile.) This pile will be both players' decks.
• Put all your lands in a big pile (face up so you can tell the difference). This is the land deck. Whenever a player does something to a deck, including drawing cards, they get to choose the land or the non-land deck. (This includes drawing your opening hand.)
• All lands tap for any color of mana and have all basic land names and types. They basically count as any basic land all the time.
• Players share a graveyard.

Why I like it:

• Basically no setup
• Minimal mana or color problems
• Optimized for things happening during the game, rather than i.e. skillful play, deck-building interestingness

I expect that this will be my default way to play in the future; it's awesome!

I thought that playing with a shared deck of cards selected kind of at random was called "Wizard's Tower" (because of the tall shared deck), but it looks like I'm wrong; I vaguely recall a similar concept called "Mass Magic" or something, but a bit of Googling gives me nothing. Dan and I added the separate land deck after the first couple of games, but it turns out that there's a format with basically these rules called fat stack. The "every land counts as all basic lands" doesn't appear in any shared-deck formats that I found, but I wouldn't be surprised if someone else thought of it first; it's pretty obvious.

I think I'll make my Star Wars card game (a project I haven't blogged about, but that I probably will blog about in the future) like this; it's a more streamlined experience, and realistically nobody's going to be building decks for my hobby game design project anyway :)

Langton's ant

I've been thinking on and off about Langton's Ant. It bugs me that nobody's resolved the conjecture about the ant eventually building a highway (I'm going to assume you looked at the wikipedia page, and not explain things again here!). I thought it'd be a fun recreational math problem to poke at, and I can now at least say that it is fun!

Today, I started out wanting to implement Langton's Ant in javascript, just to have something to play with. Implementing things like this is always interesting, because there are typically a lot of ways to write the program that are kind of annoying and painful; for example, in this case, you could encode the 2D grid the ant lives on as an array, but arrays in javascript have finite size, and so you'd have to watch closely to make sure the ant doesn't run off the edge of the array and then resize it... ugh.

Instead, I thought I'd just keep a list of locations that the ant has visited. Then, you can calculate the color of the ant's current location by checking how many times the ant's been there before. However, since the ant's rules are written from the perspective of which way the ant is facing ("turn left" or "turn right"), deciding whether the ant moves up, down, left, or right depends on the last two moves and also the color of the square. This is only mildly annoying to code, but I'm very lazy, so I kept trying to cut down the complexity.

Next, I thought I'd keep a list of move directions, like "uldrurdrd...". This would make calculating the next step easier. Interestingly, with this representation, you don't need to keep a list of positions at all -- all of the information about which squares the ant has visited is encoded in the sequence of moves! To know the color of the current square, count the number of times it has been visited before, which is equal to the number of prefixes of the move sequences where (number of up moves minus number of down moves) and (number of right moves minus number of left moves) are the same as these properties of the whole string. I wrote a (probably buggy) program based on this idea (see end of post).

This got me thinking, though, about the overall problem of the highway conjecture. In this representation, the highway appears as a repeating sequence of (up, down, left, right) moves. This is kind of nice, because it makes it easier to see when the ant has hit the highway; this sequence is kind of like an attractor, or something.

We now have, instead of an ant on a grid, a system for generating the next letter in a sequence. Given a list of letters, the next letter is determined by:
- The last letter
- The parity of the number of substrings with the same properties (u - d) and (r - l) as the overall letter sequence

This system could be generalized in a variety of ways. Using e.g. modulo 3 or 4 instead of parity would add more colors, and it's interesting to me that we could use properties other than (u - d) and (r - l) -- for example, ratios or more complex expressions.

I'm not sure, but I feel like this is incremental progress toward solving the conjecture, which makes the whole thing feel worthwhile :) If I can make more progress, you'll hear about it in a future post.

A final note: Langton's Ant is reversible, so theorems that hold into the future also hold into the past. For example, not only will the ant get arbitrarily far away from its starting point (see wikipedia), it also came from arbitrarily far away if you assume it's been running forever. If the highway conjecture is correct, ants not only eventually build highways, they came from highways, unwinding them from the infinite distance until they got to their starting point! That's kind of neat.

⁂

The program:

function move(ms) {
 possible = ({u:"lr", d:"rl", l:"du", r:"ud", "":"ud"})[ms.slice(-1)];
 x = ms.split("r").length - ms.split("l").length;
 y = ms.split("u").length - ms.split("d").length;
 console.debug(x+", "+y);
 color = 0;
 ms.split("").forEach(function (c) {
  x = x - (c=="r"?1:c=="l"?-1:0);
  y = y - (c=="u"?1:c=="d"?-1:0);
  if (x == 0 && y == 0) color = -color+1; // wrong here to include last character? Does this just invert?
 });
 return ms + possible.charAt(color);
}

Non-post for March 12

I played Magic with friends instead of writing a blog post today! I regret nothing.

Projecting maps by travel time

Maps where something else is used instead of distance or area (e.g. population, GDP, travel time) are called cartograms. Apparently maps where distances correspond to travel times are called linear cartograms, but this seems like kind of a bad name. I'll call them travel-time cartograms instead.

You could think about different parts of the 2D surface of the Earth as having different speeds when you cross them by some mode of travel (trails are easy to walk on, mountains are hard to walk up but easy to walk down, and water can't be walked on at all), and try to make a map that way. The mountain case tells us that when travel time depends on direction, a travel-time cartogram can't be made, i.e. the distance between the top and bottom of a mountain must be two things at once. I guess you could take an average, but I don't like that much -- it seems to destroy or hide a lot of information. Instead, I'll consider different modes of travel, and assume that direction doesn't matter.

I think three modes of travel (plane, car, walking) might be the sweet spot in terms of accuracy vs. difficulty, but two modes of travel is complicated enough to bring out the conceptual issues, and airplanes change our travel time around the world much more than cars do. Airplanes are pretty weird -- do you consider the points along the plane's trajectory to be quickly reachable, or do you imagine the plane disappearing from one airport and then reappearing at the other end? I'll use the time-delayed teleportation model, since I can't really get places by parachuting out of a plane. So, my first-approximation travel-time cartogram should have this property:

The distance between any two points is the minimum travel time by a combination of air travel and "land travel" (a continuous form of travel at 30 miles per hour, a rough average of car and foot travel)

What happens when I try to build this map?

• Major cities and their neighborhoods should be near each other.
• Far from airports, the map should approximate a globe (since only land travel is relevant there).
• Any shortest line between two points (along the surface) should correspond to the shortest trip between them (i.e. it should point-for-point cover the same route as the shortest trip).
• There are points on the surface between the major cities that don't correspond to points on the globe, so that travel between airports isn't immediate, but you also can't parachute out of the planes.

Instead of working with a globe, let's start with a circle (a slice of the sphere, where travellers can move only along the circle between points) and add two airports that can be moved between quickly. What we need to do, it seems to me, is bend the circle through the third dimension until the airports are near one another in 3D space, then add a line between them of length equal to the travel time. It's important to note that distance on this cartogram isn't measured in raw Euclidean terms, but instead by distance along the surface itself (just like on a globe, you measure distance along the surface instead of tunneling through the earth). If you add more airports, then it's like folding the circle up so that many points on its circumference nearly meet.

In fact, if we assume that airplanes all travel at the same speed along this circle, then you can make a separate surface corresponding only to air-travel, with points corresponding to airports and edges corresponding to flights, and this surface will be curved overall like a circle. So, we have a large "land-travel circle" and a smaller, nodes-and-edges "air-travel sphere", and we can make the final map by folding the larger circle through 3D space so that its airports meet the smaller circle's airports.

To get back to the real world, "all you have to do" is make a land-travel globe (which looks like a normal globe), an air-travel "globe" (a smaller web of connections between airports that is overall curved like a sphere), then fold the land-travel globe through the fourth dimension so that its airports meet up with the air-travel globe's points. That's pretty awkward, because now we have a 4D map that is going to be really hard for humans to read and get intuitions about!

To flatten a globe into a map in a way that lets humans understand distances, we sometimes put a grid on the sphere as guidelines for humans. Can we do a similar thing here -- put a grid on our 4D map, cut and flatten it, and then print it out? Not sure, but that seems like what we want to do!

Lots of presents

I just noticed a funny thing. I'm traveling for work, and I'm very picky about what I bring; I'm a light packer (one shoulder bag for a week trip), but I like to be as prepared as possible. Here are some of the things I have with me:

• Timbuktu bag (present from my parents)
• Down vest to stay warm (present from my parents)
• Rain jacket (present from my parents)
• Spacepak clean/dirty clothes bag (wedding present I think?)
• Socks (present from Killian's parents)
• Headphones (present from Killian)
• More socks (present from Killian's parents again!)
• Peacock-feather-pattern dress shirt (present from my parents)

Isn't that nice? I think the majority of the things I have with me are presents from someone! And I'm staying with friends, so that's like a present, too. I have a reputation for being hard to buy presents for, but apparently people are doing great at it!

Too busy to blog again!

...however, I feel a little better about writing a post like this than missing a day entirely! (It's after midnight, but this counts for March 9th in my book.)

Instead, read this post by Jeff: when do lilacs bloom?

Instead of posting today...

... I messed with the template to show all of my labels in The Archives (to the left)! Maybe I'll have time to write a post later as well.

What if life arose in Conway's Game of Life?

This is a question that I've been thinking about off and on for a while, and I think it's a really interesting one. If I had an extra career's worth of time that I could do whatever I wanted with, I can easily imagine spending it on this! I really hope that I can make progress on it one day.

For now, I guess I'll write a few blog posts. I'd like to write a longer essay eventually, and this seems like a good way to write it out piece-by-piece.

A note on philosophy of blog-posting: I could spend a lot of time explaining or linking to Conway's Game of Life, the history of people thinking about life arising in Life, and all the nice progress that's been made lately (check out these forums, the Gemini "replicator", and Golly), but that's not much fun for me. So, I won't! Sorry, essay etiquette.

A spacefiller ("Max") found in 1995

Here are the thoughts about life in Life that I think it makes sense to explain first:

Fundamental laws, higher-level laws, and simulations

Life's rules can be thought of as being like the laws of physics. However, as far as we know, living organisms under our laws of physics only arise at scales orders of magnitude larger than the scale of the most "natural" entities under our laws of physics, and function mostly on the higher-level laws of chemistry and thermodynamics. That could be true for Life, too -- maybe the smallest organisms are astronomically large, and function mostly on higher-level laws that we've yet to find. (What could these laws be? I'd love to know!) However, for the purposes of this post, I'll be assuming that life in Life isn't like this, and functions on scales where the rules of Life are relevant. I think at least some of my arguments will apply at any scale, but I'm not sure.

An interesting sub-possibility is that "life" in Life occurs most frequently within simulations run on computers that naturally occur in Life, instead of in the "basement" laws -- for example, maybe it's easier to build a computer in Life that simulates our own laws of physics (which eventually give rise to life, at least in some cases) than it is to build a functional organism under Life's rules. Again, I'll assume that this isn't true, but it'd be pretty neat if we could show that it was!

Cosmology

I'm interested in when life "naturally" arises. However, Life's rules don't specify a start state, and so there's no built-in "cosmology". The setting that seems most natural to me is to start out with a random setting of each cell in an infinite plane to On or Off, with p being a "cosmological constant" that determines the probability of a cell starting On. (Maybe intelligent organisms will later be able to experimentally determine p by examining their world?) Since different p values may be more hospitable to life than others, I'd also like to let p vary gradually over the infinite plane, sort of like the different physical constants in the level-2 Tegmark multiverse.

A not-atypical view of a small region with a random starting configuration

What is life anyway?

I'm going with a pretty basic definition:

In Conway's Game of Life, a region of cells is alive if, when placed in some "reasonably natural" environment, produce two or more copies of itself.

You'll note that this definition is imprecise. I'm also leaving out some other common criteria like homeostasis, metabolism, and growth, because I think they might look quite different in Life than they do in our physics.

Abiogenesis

...just a teaser! I'll get to this next in another post.

(This space intentionally left blank)

Not enough capacity to post today -- feeling a little exasperated, and I still have work to do today. Sorry, post-a-day challenge!

Should we bite Occam's Bullet?

(Close-second title: Does Occam's Razor cut too deep?)

(I told Amanda I'd post some philosophy stuff, but I've spent most of my posts this week on AI because I'm certifiably obsessed. So, here's a philosophy thing, and I'll leave out the application to AI for variety.)

I'm a little perturbed about using Occam's razor as a foundation of epistemology, especially in its computational forms. Here's the kind of reasoning I'm concerned about:

Physics test: If Michael Jordan has a vertical leap of 1.29 m, then what is his takeoff speed and his hang time (total time to move upwards to the peak and then return to the ground)?

Student: According to the simplest explanation, Michael Jordan has formed randomly from thermal or quantum fluctuations, and the small bubble of order he inhabits will collapse back into background heat long before he touches the ground.

You would probably not get extra credit for rigor or consistency in answering this question!

My basic worry is that the simplest explanation for a set of observations may be something that doesn't fit with any of my normal beliefs about my situation. This is because simple explanations can expand into vast universes, and in these universes there could be many instances of my circumstances (or something observer-independent, like the Michael Jordan problem above) that are nothing like what I believe to be my current situation; they could be in simulations, part of programs numerating all possible computations in order, fluctuations of some very long-lasting, near-equilibrium cosmological state, or something stranger.

(I don't think the problem goes away when you consider the set of all explanations compatible with observations, weighted by their simplicity, but I might be wrong.)

Of course, people could just as well have had my complaint when physics was just being discovered; our view of what the universe is and our place in it would probably appear extremely weird to them, violating many of their normal beliefs about their situation. Heck, the implications of quantum physics are weird enough to me now. So maybe I'm just being stubborn, and I should bite Occam's bullet and think that most of my normal beliefs about my situation are wrong.

So why don't I think we should use this kind of reasoning? I could have epistemic or instrumental reasons, I could actually be asking a different question from "what is the most likely explanation", or I could use some kind of anthropic reasoning.

• Epistemic: I don't feel like I really believe that the most likely explanation is that I'm a Boltzmann brain; I feel like I have evidence that says otherwise. However, that evidence could be fabricated, which is a big problem -- I may just have an unjustified belief that I'm not a Boltzmann brain! Should I bite Occam's bullet?
• Instrumental: if I am in a Boltzmann brain, things I do matter only over very small timescales (until the bubble collapses).
• Different question: maybe instead I want to know something like "conditioning on some other assumptions (like that most of my evidence is "real", whatever that means), what is the most likely explanation?" This actually doesn't seem so bad; it's the most appealing answer to me at the moment.
• Anthropic reasoning: I'm not particularly satisfied with this, because I'd like questions about situations without observers -- e.g. physics problems like the Michael Jordan problem above (well, versions without MJ the observer!) -- to have "reasonable" answers, instead of silly ones. In fact, that might be the most interesting part of this post -- that these problems seem like they can't be answered fully by anthropics, if we want to answer observer-free questions "sensibly".

I do like the idea of re-framing the basic epistemic question ("what is the best explanation for x, and what does this imply we should expect in x's future"), but I'm not sure where to go from there. Perhaps in future posts!

Updated version of an AI decision theory problem

Let's say we have a model-based RL system in a peculiar episodic environment: at the start of each episode, the system is copied onto another computer, and the copy's actions matter for what happens in the environment (e.g. they are playing some kind of game).

A "correct" model (state, action) → (new state) seems like it should have two properties:

1. If something causes the copy to behave differently from the system -- e.g., the copy is made incorrectly, the computer the copy runs on malfunctions, or the copy is interfered with during its decision -- the model should predict what the malfunctioning copy will actually do.
2. "Otherwise", the model should predict that the system and the copy perform the same action; that is, when the model predicts the copy's action in order to predict new state, the copy's action should be identical to the model's input action.

There are two kinds of models that I think it is reasonable to imagine our RL system making: abstract-ish models and physical-ish models. It seems like abstract-ish models will do well with property 2, and physical-ish models will do well with property 1. I can't picture the type of model that will get both properties.

• When the system has a fairly abstract and non-physical model (state, action) → (new state), it can simply model the copy's action as if it directly depended on the system's action, and it will correctly predict that the system and copy will always act the same way. However, this kind of model will make wrong predictions if something goes physically wrong with the copy.
• As the system learns a model that correctly handles things going wrong with the copy, it will probably no longer model the copy's actions as directly dependent on the system's action (in part because this is an accurate model of the physical setup). However, this kind of model seems like it will not predict that the system and the copy perform the same action when nothing goes wrong with the copy, since the copy will perform the same physical steps to make its decision regardless of what the input action is.

As the system learns, if things do go wrong with the copy, it will probably get evidence that pushes it toward physical-ish models, so that it can predict those things going wrong. However, it seems like there are common cases where a model fails property 2, but doesn't receive evidence to push it to correct this. For example:

Suppose the system is considering which action to take in state s. It considers actions a1, a2, a3. Using a physical-ish model, it predicts that the copy will take action a1 independent of the system's choice, so that possible (system choice, copy choice) pairs are (a1, a1), (a2, a1), (a3, a1), the second and third of which are mispredictions. If the system prefers the state resulting from (a1, a1) to the states resulting from (a2, a1) or (a3, a1), it will choose a1, and will never receive feedback that if it had chosen a2, the copy would have chosen a2 as well, and ditto for a3.

Intuitively, the model can't learn "what would have happened" correctly, and will need to rely on generalization in order to get this right. I don't know what kind of generalization would produce a model that does this correctly. This kind of problem will be especially bad if there are equilibria that result in the system consistently choosing the same action, as in the Prisoner's Dilemma (where the system never realizes that if it cooperated, the copy would cooperate as well). However, even outside this kind of problem, I still don't know how to make a model that fulfills properties 1 and 2.

This is just a toy case, and I expect analogous problems to come up in analogous situations (e.g. situations where there is not an exact copy, or where the "copy" is another system, human, or market that is learning to predict the system's actions). I don't know how much I should expect this kind of problem to come up in model-free approaches, but it seems worth looking into, and it would be disappointing if this kind of problem blocked us from using model-based approaches at all.