Architecting Machine Learning Systems: Which Parts are the Architect's Pain?

Designing machine learning applications can be challenging, especially software architectures for handling real-time sensor data processed by compute- and software-intensive machine learning applications. This paper answers the two research questions: 'Which parts of a machine learning pipeline do novel software architectures and framework optimize?' and the associated question of 'Which parts do the architect then need to focus on?'. The presented experiences and experimental results suggest that novel software architectures and frameworks optimize the learning and classification part of the pipelines. Therefore, the architect in particular needs to focus on data distribution and preprocessing as these parts were observed to have an overlooked computational cost and complexity. These results are important for software architects to become better at architecting machine learning-based systems.