Totally valid observation, yeah you do see it, you are right it's not all that often and you almost never see a hot young girl with a broke older guy. Attraction is an odd thing. A mate of mine, good looking guy, built like a brick shithouse, genuinely loves older girls. 40 is a minimum for him (he's just turned 30). Every time he introduces me to a new squeeze I am baffled as they are invariably just ok looking, 15 years his senior and not in peak fitness, but he just loves them and good luck to him. Rare as hens teeth I know, but we're all attracted to something different.
I suppose it is subjective as well as to what constitutes "older". When I was 25 I briefly dated an 18 year old and her friends thought I was an antique! I'm guessing we're talking 50+ here as 'older men'?