Losing weight in middle age might mean losing not just unwanted fat, but also precious bone density, at least for women, a new study suggests.

Regardless of the types of foods or amount of calcium in their diets, middle-age women who lost a moderate amount of weight over a two-year period also lost more bone density than men or younger women.

Changes in bone density after moderate weight loss might be sex-specific and influenced by hormones, the study team wrote in the Journal of Clinical Endocrinology and Metabolism.

“Weight loss has been associated with beneficial effects on cardiovascular risk factors like diabetes. However, extremes of weight loss have been associated with bone loss, and according to some studies, increased risk of fracture,” said senior author Dr. Meryl LeBoff of Brigham and Women’s Hospital in Boston.