Differentiating regulation is a promising approach to agri-environmental regulation that may potentially reduce the environmental impact of agriculture at the lowest possible costs for the farmers and society, but also possesses a number of challenges. In this article, we explore the challenges to the legitimacy of agri-environmental regulation that occurs when the regulatory regime changes from general regulation to differentiated regulation. The analysis is based on a case study of the implementation of the Buffer zone act in Denmark – a regulation that prevents agricultural production in a 10 (later 9) meter fringe around selected waterbodies. We distinguish between two different ways of legitimizing: Producing knowledge and participation. We conclude that to harvest some of the obvious benefits of differentiated regulation a number of challenges must be resolved, 1) ensuring legitimacy of differentiated regulation is crucial, 2) differentiated regulation imply that farmers are also differentiated, 3) differentiated regulation implies new uncertainties, 4) the current knowledge regime need to be reconfigured, 5) stakeholders feel that they are unevenly treated and 6) it is difficult to establish a win–win solution for all farmers on an individual level.