Obesity is heightened during aging, and although the estrogen receptor α (ERα) has been implicated in the prevention of obesity, its molecular actions in adipocytes remain inadequately understood. Here, we show that adipose tissue ESR1/Esr1 expression inversely associated with adiposity and positively associated with genes involved in mitochondrial metabolism and markers of metabolic health in 700 Finnish men and 100 strains of inbred mice from the UCLA Hybrid Mouse Diversity Panel. To determine the anti-obesity actions of ERα in fat, we selectively deleted Esr1 from white and brown adipocytes in mice. In white adipose tissue, Esr1 controlled oxidative metabolism by restraining the targeted elimination of mitochondria via the E3 ubiquitin ligase parkin. mtDNA content was elevated, and adipose tissue mass was reduced in adipose-selective parkin knockout mice. In brown fat centrally involved in body temperature maintenance, Esr1 was requisite for both mitochondrial remodeling by dynamin-related protein 1 (Drp1) and uncoupled respiration thermogenesis by uncoupled protein 1 (Ucp1). In both white and brown fat of female mice and adipocytes in culture, mitochondrial dysfunction in the context of Esr1 deletion was paralleled by a reduction in the expression of the mtDNA polymerase γ subunit Polg1. We identified Polg1 as an ERα target gene by showing that ERα binds the Polg1 promoter to control its expression in 3T3L1 adipocytes. These findings support strategies leveraging ERα action on mitochondrial function in adipocytes to combat obesity and metabolic dysfunction.