I've played varsity football, soccer, and golf, and had the opportunity to play basketball but chose not to because the coach is a cock.
Golf is the easiest to do ok in, but to be number 1 takes an incredible amount of skill. Notice I said skill though, not athletic talent...it's more a game than a sport in this respect.
Football is just really damn easy....requires almost no endurance since you break between every play.
Soccer was probably one of the hardest. Takes insane amounts of athletic ability and endurance.
Basketball was really hard, took lots of athletic ability. Seeing my friends who went through with it and played varsity, they are all in fantastic shape no homo, and it is one of the toughest sports.
I played baseball for like 8 years too. It is not easy. HItting a fastball or pitching accurately over multiple innings is tough. It is NOT, however, a test of athleticism. As has been previously stated, there are many unathletic professionals. In this respect, it's more of a game than a sport.