HEM red line

So it seems like almost everyone has a fundamental misunderstanding about how the HEM red line works and what it does/doesn't do (and to be honest, I didn't really understand it myself until recently).

Common statements seen regularly on the 2+2 forums include:
"The red line is wrong because it is based on ICM, and ICM isn't correct."
"The red line is wrong because it doesn't take into account coolers or other forms of luck."
"The red line is wrong in merge sngs because it uses 65/35 payoffs while actual payoffs are 70/30."

This post will be pretty informal, but hopefully will convince you why all of these statements are essentially wrong. I say "essentially" because I will be ignoring card removal in the analysis; but card removal plays a pretty minor role in general, and it seems reasonable to ignore it.

Now, when assessing an EV-estimating function like the HEM red line, it will be useful to use two metrics:
1) Bias. An unbiased estimator of actual EV would have the same expectation as actual profits, and would equal actual profits in the limit.
2) Variance. A lower variance estimator will generally have smaller swings, and will make it easier to assess our performance with fewer samples.

Obviously we would like an estimator that is unbiased and low variance (there would be no point in using the red line if it didn't have lower variance than the green line/actual profits).

Now let's recall how the red line works in sngs. For each hand in which two players are all-in, an "EV diff" is computed. The hand EV diff equals the difference between the expected ICM payoff over the allin outcomes, and the ICM payoff of the outcome that actually occurs (so it will be negative when you are "lucky" and positive when you are "unlucky" with respect to ICM). If no two players are allin in a given hand, the hand EV diff is 0.

Here is an example: suppose 1000 starting stacks, $100 pre-rake buyin, 6max superturbo sng, 65/35 payouts, blinds at 50/100.
Initially the ICM value is $100 to everyone.
Suppose you get in a 60-40 in the first hand and win (SB vs BB).
The ICM value of a win is $186, while that of a loss is $0.
So your expected ICM payoff is 0.6*$186 + 0.4*$0 = $111.60
The ICM payoff the actual outcome (you winning) is $186.
So the hand EV diff will be $111.60 - $186 = -$74.4

Let EV-diff-total equal the sum of all the individual hand EV diffs. The total red line EV of the tournament will equal EV-diff-total plus your actual profit (i.e., the green line). So if your EV diff total is -$50 and you get a payout of $150 in the tournament, your red line will go up by $100 and green line will go up by $150 (minus the buyin + rake, for both lines).

So to recap, we have:
Red line EV = EV-diff-total + Green line EV

Let's call the green line EV our "WINNINGS," the red line our "SKILL" and the negative of the EV-diff-total our "LUCK."
So we have
WINNINGS = SKILL + LUCK
It should also be pretty obvious that E[LUCK] = 0, since it is solely due to the chance outcome of the allin (ignoring the card removal effect of other players who have folded). So we have E[WINNINGS] = E[SKILL]; i.e., the red line is an unbiased estimator of the green line. Interestingly, this doesn't really depend on whether ICM is "correct" or not; we would have E[LUCK] = 0 for any mapping, even really naive ones.

Let's consider the same 60-40 example from above, but assume we use an arbitrary value mapping that gives payoff P of winning the allin (and $0 for losing it). Then our expected payoff will be 0.6P. With probability 0.6, we will win the allin, and the EV hand diff will be 0.6P - P = -0.4P. With probability 0.4 we will lose the allin, and the EV hand diff will be 0.6P - 0 = 0.6P. So the expected value of the hand diff equals 0.6 * (-0.4P) + 0.4*(0.6P) = 0, and therefore E[LUCK] will equal 0. This is just one example, but the same reasoning will apply in general to arbitrary value mappings.

A really dumb value mapping however could produce a very high variance, which might make the red line less useful than the green line. However, any value mapping would produce an unbiased estimator of our actual EV; so for example, using 65/35 ICM payoffs when actual payoffs are 70/30 still produces an unbiased red line, which almost definitely has significantly lower variance than the green line. A better value mapping would just decrease variance, not bias.

So in conclusion, the current HEM red line algorithm (using ICM) is unbiased (ignoring card removal), but not necessarily minimal variance. Nonetheless, it has significantly lower variance than actual EV (the green line), and so is generally a very useful tool for evaluating performance.

Thanks to Prof. Mike Bowling at Alberta, and jukofyork from the 2+2 forums for helping me understand this better.