Romy Müller

Deep neural networks are powerful image classifiers but do they attend to similar image areas as humans? While previous studies have investigated how this similarity is shaped by technological factors, little is known about the role of factors that affect human attention. Therefore, we investigated the interactive effects of task and image characteristics. We varied the intentionality of the tasks used to elicit human attention maps (i.e., spontaneous gaze, gaze-pointing, manual area selection). Moreover, we varied the type of image to be categorized (i.e., singular objects, indoor scenes consisting of object arrangements, landscapes without distinct objects). The human attention maps generated in this way were compared to the attention maps of a convolutional neural network (CNN) as revealed by a method of explainable artificial intelligence (Grad-CAM). The influence of human tasks strongly depended on image type: for objects, human manual selection produced attention maps that were most similar to CNN, while the specific eye movement task had little impact. For indoor scenes, spontaneous gaze produced the least similarity, while for landscapes, similarity was equally low across all human tasks. Our results highlight the importance of taking human factors into account when comparing the attention of humans and CNN.

In complex work domains, not all possible faults can be anticipated by designers or handled by automation. Humans therefore play an important role in fault diagnosis. To support their diagnostic reasoning, it is necessary to understand the requirements that diagnosticians face. While much research has dealt with identifying domain-general aspects of fault diagnosis, the present exploratory study examined domain-specific influences on the requirements for diagnosticians. Scenario-based interviews were conducted with nine experts from two domains: the car domain and the packaging machine domain. The interviews revealed several factors that influence the requirements for successful fault diagnosis. These factors were summarized in five categories, namely domain background, technical system, typical faults, diagnostic process, and requirements. Based on these factors, we developed the Domain Requirements Model to predict requirements for diagnosticians (e.g., the need for empirical knowledge) from domain characteristics (e.g., the degree to which changes in inputs are available as domain knowledge) or characteristics of the diagnostic process (e.g., the extent of support). The model is discussed considering the psychological literature on fault diagnosis, and first insights are provided that show how the model can be used to predict requirements of diagnostic reasoning beyond the two domains studied here.