Update 10/23/2017: This paper received the “Best Published Paper Award” for all papers published in Information Systems Research in 2016. See story here.
Our study, “More Harm than Good? How Messages that Interrupt Can Make Us Vulnerable,” has been accepted to the special issue on “Ubiquitous IT and Digital Vulnerabilities” at Information Systems Research, one of the premier journals of the field of information systems.
In the article, we examine how security messages are impacted by dual-task interference (DTI), a neural limitation in which even simple tasks cannot be simultaneously performed (i.e., multitasking) without significant performance loss. We demonstrated this in two experiments: one using fMRI and another using users’ responses to the Chrome Cleanup Tool (CCT), a security message in Google Chrome.
In the News
- Schneier on Security
- The Register
- BYU News
- PC World
- Threatpost
- Slashdot
- Naked Security
- Yahoo Tech
- CNET
- Huffington Post
Study Summary
First, we used fMRI to show how DTI occurs in the brain when a simple memory task is interrupted with a security message. We found that neural activity in the bilateral medial temporal lobe (MTL) was substantially reduced when a security message interrupted a user in a simple memory task (a high-DTI condition), compared to when a user responded to the security message by itself (Figure 1). This suggests that DTI inhibits one's ability to utilize the MTL to retrieve information from the long-term memory necessary to respond to permission warnings.
Figure 1. Increased activity in the medial temporal lobe (MTL) in response to the Warning-Only condition compared to the High-DTI condition, in which the warning interrupted a memory task. Warm colors indicate increased blood flow.
Further, we showed that the change in activation in the MTL significantly predicted users' disregard of the security message, which we define as behaving against the security message's recommended course of action.
Interestingly, we found that if we finessed the timing of the security message so that it was displayed between memory tasks (a low-DTI condition), then participants had more activation in the MTL as compared to the high-DTI treatment. In addition, participants in the low-DTI condition had significantly lower security message disregard compared to the high-DTI condition (8.8% vs. 22.92%).
Amazon Mechanical Turk Experiment using the Chrome Cleanup Tool
Next, applying the findings of our fMRI experiment, we performed a practical experiment that examined how DTI impacts users' responses to the Chrome Cleanup Tool (CCT), a security message in Google Chrome for Windows (Figure 2). The CCT detects if malware has tampered with the host computer and manipulated the browser or other Internet settings (Google 2015). When a problem is detected, the CCT displays a message to the user asking for permission to remove the unwanted software and restore Chrome's original settings. Although the CCT message is important, it does not require immediate attention and, therefore, can be delayed.
Figure 2. Google Chrome Cleanup Tool (CCT) message.
We collaborated with a team of Google Chrome security engineers who develop the CCT to identify low-DTI times to display security messages during the browsing experience, in contrast to high-DTI times when the user would likely be cognitively engaged in another task. These times were selected according to (1) DTI theory and the results of fMRI results of Experiment 1, (2) input from Google engineers on moments that were frequent in occurrence and generalizable across a wide variety of web-based activities and users, and (3) a feasibility assessment for implementing in a web browser.
The low- and high-DTI conditions were:
Low-DTI:
- At the beginning of starting the first task.
- After the video.
- After interacting with a website.
- Waiting for a file to process.
- Waiting for a page to load.
High-DTI:
- In the middle of watching a video.
- In the middle of typing.
- In the middle of transferring a confirmation code.
- In the middle of the movement to close the web page.
We tested each of these conditions were tested in connection with an online video categorization task using Amazon Mechanical Turk. A total of 856 Turkers participated.
The results were dramatic. Finessing the timing of when the CCT was displayed reduced the rate it was disregarded by users from 80% for high-DTI times to 36% for low-DTI times (see Table 1 below).
Table 1. Percentage of Security Message Disregard for high- and low-DTI experimental conditions.
Finally, we show how mouse cursor-tracking and psychometric measures can be used to validate low-DTI times for security messages to be displayed for other software applications and contexts.
Together, our findings show that the timing of when security messages are displayed makes a substantial difference in how users respond to them. Many security messages are urgent and cannot be delayed (e.g., browser malware warnings). However, for those security messages that are not attached to an immediate threat (like the CCT), using a timing that respects users' limited cognitive resources can significantly improve the effectiveness of security messages.
Acknowledgements:
We thank Elisabeth Morant, Adrienne Porter Felt, and Robert Shield of Google, Inc. for their collaboration on the Google Chrome Clean-up Tool experiment.
From the abstract:
System-generated alerts are ubiquitous in personal computing and, with the proliferation of mobile devices, daily activity. While these interruptions provide timely information, research shows they come at a high cost in terms of increased stress and decreased productivity. This is due to dual-task interference (DTI), a cognitive limitation in which even simple tasks cannot be simultaneously performed without significant performance loss. Although previous research has examined how DTI impacts the performance of a primary task (the task that was interrupted), no research has examined the effect of DTI on the interrupting task. This is an important gap because in many contexts, failing to heed an alert—the interruption itself—can introduce critical vulnerabilities.
Using security messages as our context, we address this gap by using functional magnetic resonance imaging (fMRI) to explore how (1) DTI occurs in the brain in response to interruptive alerts, (2) DTI influences message security disregard, and (3) the effects of DTI can be mitigated by finessing the timing of the interruption. We show that neural activation is substantially reduced under a condition of high DTI, and the degree of reduction in turn significantly predicts security message disregard. Interestingly, we show that when a message immediately follows a primary task, neural activity in the medial temporal lobe is comparable to when attending to the message is the only task.
Further, we apply these findings in an online behavioral experiment in the context of a web-browser warning. We demonstrate a practical way to mitigate the DTI effect by presenting the warning at low-DTI times, and show how mouse cursor-tracking and psychometric measures can be used to validate low-DTI times in other contexts.
Our findings suggest that although alerts are pervasive in personal computing, they should be bounded in their presentation. The timing of interruptions strongly influences the occurrence of DTI in the brain, which in turn substantially impacts alert disregard. This paper provides a theoretically-grounded, cost-effective approach to reduce the effects of DTI for a wide variety of interruptive messages that are important but do not require immediate attention.