Background Context Although previous studies have investigated the association between paraspinal muscle morphology and low back pain (LBP), the results are conflicting. Purpose This systematic review examined the relationship between size and composition of the paraspinal muscles and LBP. Study Design/Setting A systematic review was carried out. Patient Sample No patient sample was required. Outcome Measures This review had no outcome measures. Methods A systematic search of electronic databases was conducted to identify studies investigating the association between the cross-sectional area or fatty infiltration of the paraspinal muscles (erector spinae, multifidus, psoas, and quadratus lumborum) and LBP. Descriptive data regarding study design and methodology were tabulated and a risk of bias assessment was performed. Results Of the 119 studies identified, 25 met the inclusion criteria. Eight studies were reported as having low to moderate risk of bias. There was evidence for a negative association between cross-sectional area (CSA) of multifidus and LBP, but conflicting evidence for a relationship between erector spinae, psoas, and quadratus lumborum CSA and LBP. Moreover, there was evidence to indicate multifidus CSA was predictive of LBP for up to 12 months in men, but insufficient evidence to indicate a relationship for longer time periods. Although there was conflicting evidence for a relationship between multifidus fat infiltration and LBP, there was no or limited evidence for an association for the other paraspinal musculature. Conclusions This review found evidence that multifidus CSA was negatively associated with and predictive of LBP up to 12 months but conflicting evidence for an association between erector spinae, psoas, and quadratus lumborum CSA and LBP. To further understand the role of the paraspinal musculature in LBP, there is a need for high-quality cohort studies which extend over both the short and longer term.