(Note: I started this post not understanding the observation equation, but now I feel like I do. +1 for writing to understand!)
First, Bostrom states the Strong Self-Sampling Assumption (SSSA), an informal rule for reasoning that he thinks is correct:
(SSSA) Each observer-moment should reason as if it were randomly selected from the set of all observer-moments in its reference class.Sounds pretty good to me, but the devil's in the details -- in particular, what is a reference class?
Bostrom offers an "observation equation" formalizing SSSA. Suppose an observer-moment \(m\) has evidence \(e\) and is considering hypothesis \(h\). Bostrom proposes this rule for \(m\)'s belief:
\[P(h|e) = \frac{1}{\gamma}\sum_{o\in O_h\cap O_e}{\frac{P(w_o)}{|O_o\cap O(w_o)|}}\]
Okay, what does this mean? Ignore \(\gamma\) for now; it's a normalizing constant that depends only on \(e\) that makes sure probabilities add up to 1, I think. \(O_h\) is the set of observer-moments that are consistent with hypothesis \(h\), and \(O_e\) is the set of observer-moments that have evidence \(e\). So, what we're doing is looking at each observer-moment \(o\) with evidence \(e\) where hypothesis \(h\) is actually true, and adding up the probabilities of the worlds that those \(o\) live in, divided by the number of observers in that world that are members of \(o\)'s "reference class", which we still haven't defined.
Now let's look at the normalization constant:
\[\gamma = \sum_{o\in O_e}{\frac{P(w_o)}{|O_o \cap O(w_o)|}}\]
This is pretty similar to the above, but it just sums over observer-moments that have evidence \(e\). In fact, the inside of the sum is the same function of \(o\) as the inside of the sum of the first equation. In fact, I think we can sensibly pull this out into its own function, which semantically I think is something like the prior probability of "being" each observer:
\[P(o) = \frac{P(w_o)}{|O_o\cap O(w_o)|}\]
For each observer, the prior probability of "being" that observer is the probability of being in that world, split equally among all observers in that world that are in the same "reference class". This in turn lets us rewrite the observation equation as:
\[P(h|e) = \frac{\sum\limits_{o\in O_h\cap O_e}{P(o)}} {\sum\limits_{o\in O_e}{P(o)}}\]
This is useful, because it makes it clear that this is basically the formula for conditional probability!
\[P(h|e) = \frac{P(\text{observe }e\text{ and }h\text{ is true})}{P(\text{observe }e)}\]
So, now I feel like I understand how Bostrom's observation equation works. I expect that I'll mostly be arguing in the future about whether \(P(o)\) is defined correctly, and I still need to come back to what exactly an observer's "reference class" is. Spoiler: Bostrom doesn't pin down reference classes precisely, and he thinks there are a variety of choices.
