An critical assessment of the concepts used by De Azevedo et al. (2022) and the derived hypothesis
The third approach, RE4AI from De Azevedo et al. (2022), is essentially a refined version of the ECCOLA method. Thus, it is also an iterative add-on process, which is applicable for the complete development process by following the three operationalization steps (prepare, review, and evaluate) using again a deck of cards. While there are minor differences between ECCOLA and RE4AI, like considering more principles (11 in RE4AI vs. 8 in ECCOLA), the following concepts seem to be the most prominent difference.
Last Concept: Enriching Procedural Methods with Technical Tools
As in ECCOLA, the cards in RE4AI also consist of a motivation-, how to tackle the issue-, and a practical example part. Nevertheless, the note-making part is replaced by a tool-suggestion-part, which points to technical tools supporting the implementation of the ethical issue. Specifically, the current landscape of technical tools is matched with the used set of principles and listed on the respective card.
Evaluation Regarding the First Developer Challenge: Time Pressure
- The concept adopts the inherent challenge of toolkits by promoting a wrong confidence in the resulting overall ethical alignment.
- In addition, while matching tools with principles, the concept does not provide guidance on when to use the tools and which requirements and assumptions every tool makes. This could lead to mistakes and thus to subsequent changes which lead to a disproportionate time effort, see ‘change something change everything’ concept.
- In addition, the amount of toolkits are not evenly distributed among the respective cards, meaning that some cards provide pointers to multiple toolkits while others have no reference at all. This could lead to a harmful practice of ignoring certain principles, not because of their unimportance in the respective context, but rather because of missing toolkits. Ignoring important principles as a result could lead again to mistakes and thus to subsequent changes which lead to an additional time effort.
- However, pointing developer teams to suitable toolkits also increases the awareness of developers that these exist, decreasing the time- effort of developers to find those in the first place and elevating their usage.
Evaluation Regarding the Second Developer Challenge: Conflicting Approaches
- Focusing on the second developer challenge, matching principles with toolkits corresponds to the straightforward problem-solving approach of developers and to their training. This results in an very actionable outcome and thus to a high likelihood of being implemented.
Evaluation Regarding the Third Developer Challenge: Mushy Stuff
- Lastly, referring to the third developer challenge, providing developers with toolkits also reduces the abstract nature of principles by providing actionable steps.
- However, it is important to point out that toolkits only provide a simplification that can be used meaningful in very specific contexts and that this concept does not fully embrace the context-specific and conflicting nature of principles.