In this paper, we propose a method for selective monitoring and management of human muscle fatigue in human-robot co-manipulation scenarios. The proposed approach uses a machine learning technique to learn the complex relationship between individual human muscle forces, arm configuration and arm endpoint force that are provided by a sophisticated offline musculoskeletal model. The estimated muscle forces are used in the fatigue model to estimate the individual muscle fatigue levels online. Two fatigue management protocols are proposed that enable the robot to handle and reduce the human fatigue by altering the configuration of task execution. The first protocol uses optimisation technique to find the optimal position for task execution, where the fatigue-related endurance time can be maximised. The second protocol divides the arm muscles into groups and then alters the direction of endpoint force so that the fatigued muscle group can relax and the relaxed muscle group becomes active. The proposed method has a potential to enable the robot to facilitate safer and more ergonomic working conditions for the human coworker. The main advantage of this approach is that it can operate online, and that all the measurements can be performed by the robot sensory system, which can significantly increase the applicability in real world scenarios. To validate the proposed method, we performed multiple experiments with two collaborative tasks (polishing and drilling) under different conditions.