Numerous studies have shown that the COVID-19 pandemic had large influence on mental health during the first lockdown, but fewer studies have focused on the long-term influence on mental health. In a national longitudinal study, we examined mental well-being measured just before (fall 2019) and twice during (falls of 2020 and 2021) the COVID-19 pandemic. We utilized the Danish Health and Wellbeing Survey with questionnaires collected in 2019, 2020 and 2021 among the same study population consisting of 8179 persons. The outcome was mental well-being measured by the Short Warwick-Edinburgh Mental Well-Being Scale (SWEMWBS). Linear regression models were conducted to evaluate change in SWEMWBS from 2019 to 2021. The SWEMWBS distribution was similar in 2019, 2020 and 2021, although the distribution moved to lower scores in 2020 compared to 2019 and moved slightly to higher scores in 2021 compared to 2020. Mean SWEMWBS decreased from 24.8 (95%CI 24.7–25.0) in 2019 to 24.1 (24.0–24.2) in 2020 and increased to 24.4 (24.3–24.6) in 2021 (p < 0.001). The mean decrease from 2019 to 2020 and increase from 2020 to 2021 was strongest among women, persons below age 75 years, persons without depression and among persons with higher education and with employment. In conclusion, we find that mental well-being decreased from 2019 to 2020 and slightly increased from 2020 to 2021 without reaching the pre-pandemic level. These changes are statistically significant but small and support that COVID-19 may only have had a small long-term influence on mental health in the general population.