Why do we still vary?

I notice that last summer Karl Smith asked "Why Are There Short People?" His logic is pretty good, except for the fact that the fitness variation seems to be much starker in males than females (there is some evidence I've seen that shorter women can be more fertile, though that's balanced by the fact that larger women seem to be able to manage gestation better). In any case, height seems to be a fitness enhancing trait which is highly heritable, and yet the variation in height remains! Karl's readers offered some reasons. What do you think? Mind you, something which immediately comes to mind is that the logic presented for why everyone should be tall and vary only a touch is logic. Not all the assumptions need to hold. For example, has the advantage to height been invariant at all times and places? I have posited for example that the ...