Layered double hydroxides (LDH) and their magnetic composites have been intensively investigated as recyclable high-capacity phosphate sorbents but with little attention to their stability as function of pH and phosphate concentration. The stability of a Fe3O4@SiO2-Mg3Fe LDH P sorbent as function of pH (5–11) and orthophosphate (Pi) concentration (1–300 mg P/L) was investigated. The composite has high adsorption capacity (approx. 80 mg P/g) at pH 5 but with fast dissolution of the LDH component resulting in formation of ferrihydrite as evidenced by Mössbauer spectroscopy. At pH 7 more than 60% of the LDH dissolves within 60 min, while at alkaline pH, the LDH is more stable but with less than 40% adsorption capacity as compared to pH 5. The high Pi sorption at acid to neutral pH is attributed to Pi bonding to the residual ferrihydrite. Under alkaline conditions Pi is sorbed to LDH at low Pi concentration while magnesium phosphates form at higher Pi concentration evidenced by solid-state 31P MAS NMR, powder X-ray diffraction and chemical analyses. Sorption as function of pH and Pi concentration has been fitted by a Rational 2D function allowing for estimation of Pi sorption and precipitation. In conclusion, the instability of the LDH component limits its application in wastewater treatment from acid to alkaline pH. Future use of magnetic LDH composites requires substantial stabilisation of the LDH component.