Fitness is a curve. You can be Lance Armstrong or you can be really out of shape at the opposite end. People enter the curve wherever they are and then they can move up the curve by better nutrition and better exercise.
I had the best of both worlds when I was a kid. I'd spend a quiet week with my mum then I'd go to my dad's property in the Adelaide Hills where there were all these kids and animals running around.