Objectives: Despite distinct aetiologies, the end-stages of primary osteoarthritis (OA) and secondary OA are described by common radiological features. However, the morphology of the bone–cartilage unit may differ depending on the pathogenesis. In this cross-sectional study, we aimed to investigate the histological differences in the bone–cartilage unit of the femoral head between patients with primary OA and secondary OA due to rheumatoid arthritis (RA). Method: Femoral heads were obtained from 12 patients with primary OA, six patients with secondary OA due to RA, and 12 control subjects. The femoral heads were investigated using stereological methods to ensure unbiased quantification. Results: The volume (mean difference [95% confidence interval]) (2.1 [0.5;3.8] cm3, p = 0.016) and thickness (413 [78.9;747] µm, p = 0.029) of the articular cartilage and the thickness of the calcified cartilage (56.4 [0.4;113] µm, p = 0.017) were larger in patients with primary OA than in patients with secondary OA due to RA. Femoral head volume (1.2 [−3.6;6.1] cm3, p = 0.598), bone volume fraction (−1.1 [−2.8;5.1] cm3, p = 0.553), subchondral bone thickness (−2.5 [−212;207] µm, p = 0.980), and osteophyte area (25.3 [−53.6;104] cm2, p = 0.506) did not differ between patients. Conclusion: The thicker calcified cartilage in primary OA preceding the loss of articular cartilage can be attributed to endochondral ossification. Patients with secondary OA due to RA had severely thinner calcified cartilage as the pathogenesis is driven by inflammation and is characterized by a generalized and more severe loss of articular cartilage.