The sportman/woman

I've often wondered if I'd actually had time at school away from being a geek in the wrong social groups and actually taken the time to do more sport would I have been any good??

This may be a random thought process, after all it's been nearly 9 years since I left secondary school and a long time since I embraced my geekdom but today got me thinking.

Myself and the 'wife' played squash today, we've started to play once a week and it's a good way to get some exercise but I often get to the point where if I'm not playing as well as I feel and I can I get really frustrated with myself. In the end I play worse for the tantrum than if I accepted that I should just relax and play. I've always thought that I just have high expectations of myself and think that I should try harder and when I struggle I blame myself and get cross with me. But recently it's come out that unless you really know me this might just come across as a big strop and very unsportsmanlike and a bit crap.

Can I still be very sportsmanlike and still have high standards of myself??
I've never managed to stick at a sport, at the moment I am a 'what if' kind of sportsman I seem to make a Very good beginner but then something gets in the way... Cricket at school- on the boys team but no one to play, squash- I didn't think I was very good so I didn't play for school more than once, rugby- played in the US but damaged my knee in the uk and now work and life stops me joining a team, fencing- knee damaged, archery- good beginner but...

So really I just find an excuse not to be good at something, I don't give it a go in case I'm crap, which is an odd juxtaposition as in my writing I still do it even if I don't think I'm very good...
Is it just a personal flaw in self confidence or circumstance that's always just stopped me finding out??