Early life Western diet (WD) consumption leads to impaired memory function, particularly for processes mediated by the hippocampus. However, the precise critical developmental window(s) during which WD exposure negatively impacts hippocampal function are unknown. Here, we exposed male and female rats to a WD model involving free access to a variety of high-fat and/or high-sugar food and drink items during either the early-adolescent period (postnatal days [PN] 26-41; WD-EA) or late-adolescent period (PN 41-56; WD-LA). Control (CTL) rats were given healthy standard chow throughout both periods. To evaluate long-lasting memory capacity well beyond the early life WD exposure periods, we performed behavioral assessments after both a short (4 weeks for WD-EA, 2 weeks for WD-LA) and long (12 weeks for WD-EA, 10 weeks for WD-LA) period of healthy diet intervention. Results revealed no differences in body weight or body composition between diet groups, regardless of sex. Following the shorter period of healthy diet intervention, both male and female WD-EA and WD-LA rats showed deficits in hippocampal-dependent memory compared to CTL rats. Following the longer healthy diet intervention period, memory impairments persisted in male WD-EA but not WD-LA rats. In contrast, in female rats the longer healthy diet intervention reversed the initial memory impairments in both WD-EA and WD-LA rats. Collectively, these findings reveal that early-adolescence is a critical period of long-lasting hippocampal vulnerability to dietary insults in male but not female rats, thus highlighting developmental- and sex-specific effects mediating the relationship between the early life nutritional environment and long-term cognitive health.