it totally agree with your weight argument... given 2 similar size bodies, the lighter one will experience a greater effect of wind resistance.
But the "higher" center of gravity thing hindering ladies in skiing is just wrong. women have a lower center of gravity than men(check it on google if you don't believe me), Not to mention, if you look at gymnastics, moguls, or aerials it docent affect them nearly as much, or so it seems.
Just a thought, idk if it makes sense, but men might be better off with their higher center of gravity considering the line is pushed towards the hips once ski boots/skis are on. ??