Failure Modes of AI Systems in Regulated Environments A Systems Architecture Perspective - Naresh Bandaru (International Journal of Intelligent Systems and Applications in Engineering, 2020)
Automated systems of artificial intelligence are gradually penetrating the controlled activities in finance, healthcare, and public services. A lot of the failures of these systems are considered as model or data failures. The thesis that is presented in this paper suggests that the majority of compliance failures are caused by flaws in the architecture of a system and not the errors in algorithms. The study is based on a quantitative, architecture-level analysis to find out the prevalence of failure modes in data pipelines, model lifecycle management, inference systems, and monitoring architectures. The findings indicate that there are evident trends between architectural design decisions and audit failures and audit governance risks. The paper brings out compliance-native architectural solutions which lessen risk by tracking, determinism and governance controls.