Lol I hope it's not sexual (though not gonna lie, horseback riding is great cross training for really good sex-but maybe that's just me). Not to mention, from having done work in breeding too...stallions are huge, causing me to cringe in so many ways at the thought of the whole donkey show thing, (bad mental image here).
I'm sure it's different for everyone, but I know a lot of girls who, like me, never quite grew out of the "I want a pony for christmas" phase haha. For me: I have always loved animals and felt connected I guess to them-they just make a lot of sense to me and I've always enjoyed learning about and training them (good thing I'm doing vet school now, seems like the right career choice for me). I was the kid who brought home injured creatures to fix up, and was really into helping take care of my pets. I started horseback riding when I was about 5-6, doing western stuff mostly, then I switched to jumping when I was about 9. I work mostly with eventing horses now but I still ride both English and Western. It's fun, it's good exercise, and it's truly a team sport if done right (except your partner is way bigger, faster, and not as smart). It's really wonderful to put the training into the horse and yourself and improve together, though even sitting a horse for the first time and going for a good ride can be fantastic (I mean horses naturally want to graze and migrate around open range in herds, yet pair them with a good rider and they'll jump huge fences, chase cattle, perform pretty amazing moves such as in dressage or reining, run 100 miles on gnarly trails, etc. Not many other animals do anything close to that kind of athletic level, maybe some dogs). Not to mention I would describe the exhilaration felt from a good gallop as on par with the feeling of powder day skiing or hucking some awesome cliff for the first time.
Besides, I bet most cat/dog owners on here love their pets. I absolutely love my border collies too. It's no different, except the pet horse that you love also helps you do a sport, or work if you are a rancher.
Maybe more detail then you want, but that answers the question at least :)