Taihangia rupestris Yu & Li, an early spring flowering plant of the Rosaceae family, is classified as a second-level protected species in China. Endemic to the cliff faces of the Taihang Mountains in central China, this rare species requires a comprehensive understanding and conservation approach. In this study, we analyzed population-level genetic variations in both chloroplast and nuclear genomes. Taihangia rupestris var. ciliata exhibited higher genetic diversity than T. rupestris var. rupestris in both genome types. All individuals of T. rupestris var. rupestris formed a single cluster, while individuals of T. rupestris var. ciliata were grouped into three distinct clusters. Approximately 29.82 Ma, T. rupestris diverged from its closely related species. Taihangia rupestris var. ciliata appeared first, followed by the differentiation of T. rupestris var. rupestris, which adapted to the climate of the southern Taihang Mountains, around 3.55 Ma. Additionally, our study identified several chloroplast genes potentially involved in variety adaptive differentiation. These findings enhance our understanding of environmental adaptation and differentiation in the two T. rupestris varieties, providing valuable genetic insights for the conservation and utilization of this species.