Vive la difference.
There's no short answer here. Some Australian data which might point the way. Overall, women participate more than men in exercise, recreation and sport. For aerobics/fitness activities, the figures are 23% for women and 15% for men.
Constraints to participation: Whilst things like age, injury, lack of interest are equally reported by men and women; 27% of men cite lack of time due to work and study as the reason (women 17%) and family commitments 5% (women 13%). So, more men than women are in the workforce, so more constrained to particpate.
Motivators. Men who participate in sports and physical recreation are motivated by health and fitness (50%) and enjoyment (27%). Women on the other hand report these at 58% and 16%.
What's the decision making process? A model (by no means the only one) might be:
1) Am I able (a perception of competence); then
2) Is it worth it (enjoyment, beliefs, attitudes); then
3) Enablers (such as fitness, skills, access, safety, the environment); then
4) Re-inforcers (such as role models, encouragement, support).
Since 'enjoyment' is a significant factor for men, there are a lot to "lose" at that second decision-making point (not to mention the first if the exercise involves 'complicated choreography').