In future smart communities, buildings must possess energy efficient systems, provide absolute comfort, empower occupants with information, adapt operations to energy grid situations and facilitate strategic maintenance. A key condition to realize these deliverables, is a proper designed and well-equipped building automation and control system (BACS). Thus, long-term solutions call for multi-criteria approaches capable of assessing and evaluating the building automation systems' performance based on multiple impacts. This paper introduces an innovative holistic tool ‘IBACSA’, for BACS assessment and smartness evaluation using a qualitative-based multi-criteria approach. A specific set of relevant systems and their associated services form the basis of the proposed auditing and evaluation methodology. The targeted building domains comprise 60 different services in total based on guidelines and functionalities listed in the EN15232 standard. The proposed tool is able to quantify the specific impacts of the selected control functionalities based on five criteria using the points grading principle. IBACSA demonstration in a case study university building is reported and evaluated. The building overall scored 79% on energy efficiency, 64% on maintenance & fault prediction, 24% on energy flexibility & storage, 70% on comfort and 84% on information to occupants. The implementation of IBACSA in the case study shows the potential of the tool in auditing various BACS features and reporting the impact of upgrades and retrofits. The proposed tool serves as a comprehensive and user-friendly instrument in a systematic building initial and retro commissioning platform, ensuring a good start-up for newly built and retrofitted energy efficient buildings.