You need to watch out for the annual jerk festival in the Horniman, and practice evasive moves to dodge the Big Issue seller at Sainsbury's and the drivers screaming down Manor Mount the wrong way!!
Seriously, I moved to Forest Hill shortly after coming down from Scotland in 1985. The only disadvantage I can see is that it can be a pain to get to the M40 or M1 or M11 if you make frequent trips north.
But there are lots of pluses. For me the main one has to be that living on the hill (and I live pretty much at the top of it!) makes you feel like you are well out of London even though it is so close. I love the Horniman park and museum which is probably our best asset. The park is superbly maintained and a great place to sit and ponder (for the 364 days a year it is free of jerky fanatics!).
I lived in Balham (Gateway to the South) for about a year when I first came south, and would never exchange Forest Hill for there. In fact I can't think of a good reason to move anywhere else in London which is probably why I have been here 25 years!!