Considerations for the Safety Analysis of AI-Enable Systems

Abstract

This study explored the applicability of hazard analysis techniques to Artificial Intelligence/Machine Learning AI-enabled systems, a growing area of concern in safety-critical domains. The study evaluates 127 hazard analysis techniques described in the System Safety Society’s System Safety Analysis Handbook (1997) for their relevance to the unique challenges posed by AI-enabled systems. A qualitative criteria-based assessment framework was employed to systematically analyze each technique against key AI-specific considerations, including complexity management, human-AI interaction, dynamic and adaptive behavior, software-centric focus, probabilistic and uncertainty handling, and iterative development compatibility. The evaluation process involved defining criteria to address AI/ML systems' distinctive characteristics, assessing each method's applicability, and ranking techniques based on their alignment with AI-related challenges. Findings indicate that Fault Tree Analysis (FTA) and Human Reliability Analysis (HRA) are highly relevant for performing safety on AI-enabled systems. Other techniques, such as What-If Analysis, require adaptation to address emergent behaviors. This study provides a framework for selecting and tailoring hazard analysis methods for AI-enabled systems, contributing to developing robust safety assurance practices in an increasingly intelligent and autonomous era.