Longtermism and Tiny Probabilities

If you discount small probabilities to zero, should you still be a longtermist? Yes! Or so I argue in an interview I gave to Future Matters-newsletter.

Some of your research focuses on what you call “probability discounting” and whether it undermines longtermism. Could you tell us what you mean by “probability discounting” and your motivation for looking at this?

Probability discounting is the idea that we should ignore tiny probabilities in practical decision-making. Probability discounting has been proposed in response to cases that involve very small probabilities of huge payoffs, like Pascal’s Mugging.

For those who’re not familiar with this case, it goes like this: A stranger approaches you and promises to use magic that will give you a thousand quadrillion happy days in the seventh dimension if you pay him a small amount of money. Should you do that? Well, there is a very small, but non-zero, probability that the stranger is telling the truth. And if he is telling the truth, then the payoff is enormous. Provided that the payoff is sufficiently great, the offer has positive expected utility, or at least that’s the idea. Also, the mugger points out that if you have a non-zero credence in the mugger being able to deliver any finite amount of utility, then the mugger can always increase the payoff until the offer has positive expected utility—at least if your utilities are unbounded.

Probability discounting avoids the counterintuitive implication that you should pay the mugger by discounting the tiny probability of the mugger telling the truth down to zero. More generally, probability discounting is one way to avoid fanaticism, a term used to refer to the philosophical view that for every bad outcome, there is a tiny probability of a horrible outcome that is worse, and that for every good outcome, there is a tiny probability of a great payoff that is better. Other possible ways of avoiding fanaticism are, for example, having bounded utilities or conditionalizing on knowledge before maximizing expected utility.

Within probability discounting, you distinguish between “naive discounting” and other forms of discounting. What do you mean by “naive discounting”?

Naive discounting is one of the simplest ways of cashing out probability discounting. On this view, there is some threshold probability such that outcomes whose probabilities are below this threshold are ignored by conditionalizing on not obtaining these outcomes.

One obvious problem with naive discounting is where this threshold is located. When are probabilities small enough to be discounted? Some have suggested possible thresholds. For example, Buffon suggested that the threshold should be one-in-ten-thousand. And Condorcet gave an amusingly specific threshold: 1 in 144,768. Buffon chose his threshold because it was the probability of a 56-year-old man dying in one day—an outcome reasonable people usually ignore. Condorcet had a similar justification. More recently, Monton has suggested a threshold of 1 in 2 quadrillion—significantly lower than the thresholds given by the historical thinkers. Monton thinks that the threshold is subjective within reason: there is no single threshold for everybody.

Another problem with naive discounting comes from individuating outcomes. The problem is that if we individuate outcomes very finely by giving a lot of information about them, then all outcomes will have probabilities that are below the threshold. One possible solution is to individuate outcomes by utilities. The idea is that outcomes are considered “the same outcome” if their utilities are the same. This doesn’t fully solve the problem though. In some cases, all outcomes might have zero probability. Imagine for example an ideally shaped dart that is thrown on a dartboard. The probability that it hits a particular point may be zero.

Lastly, one problem with naive discounting is that it violates dominance. Imagine a lottery that gives you a tiny probability of some prize and otherwise nothing, and compare this to a lottery that surely gives you nothing. The former lottery dominates the latter one, but naive discounting says they are equally good.

Are there forms of probability discounting that avoid the problems of naive discounting?

One could solve the previous dominance violation by considering very-small-probability outcomes as tie-breakers in cases where the prospects are otherwise equally good. This is not enough to avoid violating dominance though, because the resulting view still violates dominance in a more complicated case. There are also many other ways of cashing out probability discounting. Naive discounting ignores very-small-probability outcomes. Instead, one could ignore states of the world that have tiny probabilities of occurring. The different versions of this kind of “state discounting” have other problems, though. For example, they give cyclic preference orderings or violate dominance principles in other ways.

There is also tail discounting. On this view, you first order all the possible outcomes of a prospect in terms of betterness. Then you ignore the edges, that is, the very best and the very worst outcomes. Tail discounting solves the problems with individuating outcomes and dominance violations. But it also has one big problem: it can be money-pumped. This means that someone with this view would end up paying for something they could have kept for free—which makes it less plausible as a theory of instrumental rationality.

Why do you think that probability discounting, in any of its forms, does not undermine longtermism?

In one of my papers, I go through three arguments against longtermism from discounting small probabilities. I focus on existential risk mitigation as a longtermist intervention. The first argument is a very obvious one: that the probabilities of existential risks are so tiny that we should just ignore existential risks. This is the “Low Risks Argument”. But, it does not seem to be the case that the risks are so small. Even in the next 100 years, many existential risks are estimated to be above any reasonable discounting thresholds. For example, Toby Ord has estimated that net existential risk in the next 100 years is 1/6. The British astronomer Sir Martin Rees has an even more pessimistic view. He thinks that the odds are no better than fifty-fifty that our present civilization survives to the end of the century. And the risks from specific sources also seem to be relatively high. Some estimates Ord gives include, for example, 1 in 30 risk from engineered pandemics and 1 in 10 risk from unaligned artificial intelligence. (See Michael Aird’s database for many more estimates.)

But now we come to the problem of how outcomes should be individuated. Although the risks in the next 100 years are above any reasonable discounting thresholds, the probability of an existential catastrophe due to a pandemic on the 4th of January 2055 at 13:00-14:00 might be tiny. Similarly, the risk might be tiny at 14:00-15:00, and so on. Of course, ignoring a high net existential risk on the basis of individuating outcomes this finely would be mad. But it is difficult to see how naive discounting can avoid this implication. Even if we individuate outcomes by utilities, we might end up individuating outcomes too finely because every second that passes could add a little bit of utility to the world.

I mentioned earlier that tail discounting can solve the problem of outcome individuation. But what does it say about existential risk mitigation? Consider one type of existential risk: human extinction. Tail discounting probably wouldn’t tell us to ignore the possibility of a near-term human extinction even if its probability was tiny. Recall that tail discounting only ignores the very best and the very worst outcomes, provided that their probabilities are tiny. As long as there are sufficiently high probabilities of both better and worse outcomes than human extinction, human extinction will be a “normal” outcome in terms of value. So we should not ignore it on this view.

The second argument against longtermism that I discuss in the paper concern the size of the future. For longtermism to be true, it also needs to be true that there is in expectation a great number of individuals in the far future—otherwise it would not be the case that relatively small changes in the probability of an existential catastrophe have great expected value. The “Small Future Argument” states that once we ignore very-small-probability scenarios, such as space settlement and digital minds, the expected number of individuals in the far future is too small for longtermism to be true. Again, consider tail discounting. Space settlement and digital minds might be the kind of unlikely best-case scenarios that tail discounting ignores. So is the Small Future Argument right if you accept tail discounting? No, it does not seem so. Even if you ignore these scenarios, in expectation there seem to be enough individuals in the far future, at least if we take the far future to start in 100 years. This is true even on the relatively conservative numbers that Hilary Greaves and Will MacAskill use in their paper “The Case for Strong Longtermism”.

The final argument against longtermism that I discuss in the paper states that the probability of making a difference to whether an existential catastrophe occurs or not is so small that we should ignore it. This is the “No Difference Argument”. Earlier I mentioned the idea of state discounting on which you should ignore states that are associated with tiny probabilities. State discounting captures the idea of the No Difference Argument naturally: there is that one state in which an existential catastrophe happens no matter what you do, one state in which an existential catastrophe does not happen no matter what you do, and the third state in which your actions can make a difference to whether or not the catastrophe happens. And, if the third state is associated with a tiny probability, then you should ignore it.

I think the No Difference Argument is the strongest of the three arguments against longtermism that I discuss. Plausibly, at least for many of us, the probability of making a difference is indeed small, possibly less than some reasonable discounting thresholds. But there are some responses to this argument. First, as I mentioned earlier, the different versions of state discounting face problems like cyclic preference orderings and dominance violations. So we might want to reject state discounting for these reasons. Secondly, state discounting faces collective action type problems. For example, imagine an asteroid heading toward the Earth. There are multiple asteroid defense systems and (unrealistically) each has a tiny probability of hitting the asteroid and preventing a catastrophe. But the probability of preventing a catastrophe is high if enough of them try. Suppose that attempting to stop the asteroid involves some small cost. State discounting would then recommend against attempting to stop the asteroid because the probability of making a difference is tiny for each individual. Consequently, the asteroid will almost certainly hit the Earth.

To solve these kinds of cases, state discounting would need to somehow take into account the choices other people face. But if it does so, then it no longer undermines longtermism. This is because plausibly we collectively, for example the Effective Altruism movement, can make a non-negligible difference to whether or not an existential catastrophe happens. So, my response to the No Difference Argument is that if there is a solution to the collective action problems, then this solution will also block the argument against longtermism. But if there is no solution to these problems, then state discounting is significantly less plausible as a theory. Either way, we don’t need to worry about the No Difference Argument.

To sum up, my overall conclusion is that discounting small probabilities doesn’t undermine longtermism.

Here's a link to the full paper:
PDF

The original interview with Future Matters-newsletter is here.