Internal Audit (Raji et al., 2020)

First Concept: Operationalizing Principles through Tensions In the SMACTR framework, the researchers operationalise principles by translating them into tensions, which describes a situation in which two or more principles are in conflict with each other. This is achieved by using the failure modes and effects analysis (FMEA) methodology. FMEA is a “methodical and systematic risk management approach that examines a proposed design or technology for foreseeable failures… [in order to] define, identify[,] and eliminate potential failures” (Raji et al., 2020, p. 36). It is used during the mapping, artifact collection, testing, and reflection stages and is, thus, the core method of the SMACTR framework. By following the methodology, tensions between the requirements of various stakeholders (e.g., the AI system should respect my privacy) and requirements that various components of the AI system have to adhere to (e.g., to work, the AI system has to collect and analyse as much personal data as possible), are identified. For every resulting tension, the severity, likelihood of occurrence, and likelihood of detection are determined and quantified on a scale from one to 10. These three components are multiplied to calculate the risk priority number, to prioritise all tensions, and to focus on the most important ones during the development (Rismani et al., 2021). This makes the work of Raji et al. (2020) the first to adopt the FMEA framework to identify and prioritise ethical issues in the AI system development.

Evaluation regarding the first challenge: Time pressure

• The rich methodological guidance of the FMEA methodology could potentially slow the design process and increase the development time (Raji et al., 2020). This is especially true for the initial development steps, during which a significant part of the FMEA methodology is conducted. This requires a considerable upfront investment from developer teams and causes uncertainty on whether the investment will yield results.
• However, it is essential to note that the additional time effort could be decreased to an acceptable minimum. This is because, first, work packages in the FMEA methodology overlap with tasks usually executed in the requirements engineering process of agile methodologies, such as conducting a stakeholder analysis (Sharp et al., 1999), and, second, the rich methodological guidance of FMEA characterises it as a powerful methodology with the ability to identify all potential ethical failures of an AI system, assuming that the ethical principles the AI system should adhere to remain constant over time (Li et al., 2022). A complete list of tensions at the start of the project eliminates the need for later changes, which could lead to a disproportionate time effort since changing one component will likely lead to additional changes in related components, referred to as ‘change something, change everything’ (Fu et al., 2012).

Evaluation regarding the second challenge: Conflicting Approaches

• The operationalisation of principles with the use of a widely recognised and explicit step-by-step methodology (Li et al., 2022) corresponds with the straightforward and explicit problem-solving training of developers. Consequently, this approach is a “commonly used and well-known [tool and characterised as a] safety symbol of functional safety” (Meyer & Reniers, 2013, as cited in Li et al., 2022, p. 2). The list of prioritised tensions, based on a proven set of variables, is translatable to various subsequent engineering methods, which further underlines the integrational potential with methods that are already used in the development processes.
• However, the high degree of methodological guidance can also lead to a limited scope for adjustments for developer-specific preferences and could, thus, deter developers with differing preferences.

Evaluation regarding the third challenge: Mushy Stuff

• Lastly, evaluating the concept in relation to the third challenge, ‘mushy stuff’, the operationalisation of principles as tensions is likely to remove the inherently abstract nature of principles. Especially the accompanying significant consideration of the context-specific and conflicting nature of principles is a unique characteristic of this approach (Whittlestone et al., 2019). Consequently, this concept is actionable and acceptable for developers, making it a practical operationalisation of principles.

Hypothesis Derivation

To conclude this evaluation, it is evident that this concept supports developers in all three challenges by, first, possibly limiting the overall time effort to an acceptable minimum through streamlining and synergy potentials; second, matching the working style of practitioners with the methodological guidance of FMEA; and, last, providing a specific means to operationalise principles. This makes the concept straightforward to implement and execute in an agile development process. However, it still requires an additional time effort with high upfront investment from the developer team and provides little room for developer-specific adjustments. This may lead to the perception that the concept is particularly suited to projects for which accurate execution is of significant importance, specifically those that are considered to be ‘high-risk projects’, as in projects that touch on fundamental rights of humans and businesses, such as in education, creditworthiness, and policing (Lütge et al., 2022). Concluding, this thesis hypothesises that (H1) the operationalisation of principles with tensions with the use of a method with a high degree of methodological guidance is generally perceived as useful by developers.

Second Concept: Documentation Trail Methods such as FMEA leads to the generation of an extensive knowledge base. To document this knowledge, each stage has a set of individually designed documentation templates, which, when combined, form the audit report, with the design decisions as well as the strengths and weaknesses of the AI system. With this endeavour, Raji et al. (2020) aim to provide a “’transparency’ trail of documentation” (Raji et al., 2020, p. 38), which “is intended to contribute to closing the accountability gap in the development and deployment of… artificial intelligence systems” (Raji et al., 2020, p. 33).

Evaluation regarding the first challenge: Time pressure

• Considering the fast-paced development environment of AI systems (Raji et al., 2020), the uncertain subsequent utilisation of the documentation, and the manual and operative nature of these tasks, the documentation effort might be considered an additional burden by developer teams (Aghajani et al., 2019).
• Nevertheless, documentation requirements are already a best practice in the AI system development process. Extending these requirements for the implementation of ethical principles could limit the additional effort to an acceptable minimum (Aghajani et al., 2019). This streamlining potential is further supported through individually designed documentation templates, which remove the need to conceptualize one in the first place and, thus, allows the developer team to take action directly.
• In addition, providing solid documentation and, thus, an accountability measure, impacts related principles positively. Following the argument of Morley, Floridi, et al. (2021), transparency is seen as a “second order principle, [in the sense that] it is linked with all [other] principles” (p. 2155). This means that the successful implementation of transparency increases the likelihood of successfully implementing all other principles. Therefore, a documentation trail can have far-reaching effects, beyond the implementation of one principle, transparency, and could decrease the overall effort of implementing all other principles.
• Implicit knowledge about implementing ethical principles into an AI system also increases the overall knowledge base for the company and, therefore, enhances the generation of best practices (Aghajani et al., 2019). In turn, the best practices could decrease the time effort and, thus, streamline the entire development process. The overall time effort could be reduced even further through synergy effects with subsequent processes, such as audit and certification processes that require extensive documentation.

Evaluation regarding the second challenge: Conflicting Approaches

• The concept essentially expands already used documentation practices. Thus, the described documentation concept is known to developer teams and matches their current working style.
• However, as mentioned earlier, in general, documentation processes are frequently viewed as a time intensive burden (Arisholm et al., 2006), and their degree of usefulness is highly dependent on the specific context and the up-to-datedness (Garousi et al., 2013). This is true for documentation processes in general and not specifically for the documentation of the ethical dimension. However, this indicates that ethical documentation tasks may also be considered burdensome and of only limited usefulness.

Evaluation regarding the third challenge: Mushy Stuff

• The act of documenting knowledge and making it widely accessible (between teams or companies) may decrease the abstract nature of principles. This is because accessing a knowledge base that focuses on the implementation of ethical principles provides developer teams with specific examples of what to do and what to avoid. Various specific examples would be available, which makes the underlying dimensions of a principle understandable.

Hypothesis Derivation

• To conclude this evaluation, it is evident that this concept supports developers in all three challenges by possibly reducing the overall time effort to an acceptable minimum through synergy effects with important follow-up processes (such as audit and certification) and the positive implementation effect on all other principles. In addition, this concept extends already-used documentation processes and, thus, matches developers’ working style while reducing the abstract nature of principles by enabling developer teams to access a database of specific examples. Consequently, this thesis hypothesises that (H2) the execution of an extensive documentation trail with individually designed templates following each development stage is generally perceived as useful by developers working on AI systems.