Purpose: Women with breast cancer (BC) who have a mastectomy may subsequently undergo breast reconstruction (BR). This study aimed to identify (1) factors associated with having BR, (2) factors associated with immediate BR (IBR) and delayed BR (DBR), and (3) associations between no BR, IBR and DBR and physical and mental quality of life (QoL) 5 years after diagnosis. Methods: Analyses were based on data from the national French cancer cohort VICAN, which followed a representative sample of cancer survivors, including BC survivors, for 5 years after diagnosis. BR and BR type (IBR/DBR) were identified using medico-administrative databases. The SF12 scale was used to measure mental and physical QoL. Multivariate logistic regressions were used to identify factors associated with BR, and linear models to evaluate associations between BR and BR type with QoL. Results: Of the 1192 BC survivors in VICAN, 32.6% (n = 388) had a mastectomy. Among them, 60.1% (n = 233) had BR. Of these, 38.6% (n = 90) and 61.4% (n = 143) had IBR and DBR, respectively. Compared with women who had BR, women who did not were more likely to be older and to have a lower level of health literacy. Compared with women who did not have BR, those with IBR had better mental QoL, while those who had either IBR or DBR had better physical QoL. Conclusion: Older women and those with inadequate health literacy were less likely to have BR. This may reflect women's preferences, inequalities in care options offered after a mastectomy, and socioeconomic barriers to accessing BR. These issues need further exploration. Furthermore, BR was associated with a better long-term physical QoL. IBR was associated with better mental QoL and should be promoted when possible.