Pilot 11 Updates

Authorized members of the research team can access the Qualtrics survey here. Others can see the survey contents here.

Stuart manually read all 30 responses, took notes, identified two participants to follow-up with, and removed one participant who appeared not to have participated in good faith. All messages and responses are in those notes.

New in this pilot

We added a new topic: mental-health harm.

After a large number of responses in Pilot 10 that appeared to be automated, disingenuous, or otherwise not provided in good faith, we noted in both the Prolific advertisement and the consent form that we would be manually reviewing every response before approving it and releasing compensation.

We increased the payment to $4.50 per participant as the median time to take the survey increased from 15 to 20 minutes.

Results

There were 29 participants in this pilot.

Due to a survey error, the first 10 participants were only assigned (at random) to 2 of 3 loss-focused scenarios. (When we added mental health, a 2 choose 2 to randomize order became 3 choose 2). The percents of participants who responded to these are adjusted to be among only those who were asked about the harm.

We were so surprised that over 80% of participants linked the mental health topic back to one of their three-worst tech harms that we manually checked the data. Participants 1, 7, and 8 were not asked about mental health due to the survey error impacting the first 10 participants. Of the 26 others asked, one can see from the loss stories (and the raw survey data that we double-checked) that 22 did indeed connect mental health back to their three worst harms.

Participants started by writing loss stories about the three worst technology-related harms they had experienced.

Summary Across Scenarios

TBD — The percent of *pilot* participants who reported experiencing events causing harms (the 13 bars on the left) or technology-related harms that they did not need to associate with specific events (the 3 bars on the right). Losses due to failures of security measures to protect participants from attack are paired (left bar) against harms due to security measures themselves harming participants (right bar). Each bar is broken down into colors by the Likert severity of harm each participant reported on a Likert scale.

A bar chart summarizing the percent of participants who had experienced each harm event. — The percent of *pilot* participants who reported experiencing events causing harms (the 13 bars on the left) or technology-related harms that they did not need to associate with specific events (the 3 bars on the right). Losses due to failures of security measures to protect participants from attack are paired (left bar) against harms due to security measures themselves harming participants (right bar). Each bar is broken down into colors based on whether the participant connected the experience/harm to one of the three worst experiences they described at the start of the study ("original"), whether they said they should have included the experience/harm as one of their three worst ("revised"), or whether it did not warrant a position in their top three ("not worst"). Those that had suffered the experience/harm are broken down into those who believe the harm could or could not happen to them.

When participants reported having suffered one of the described scenarios, we asked them how recently they had experienced it.

A bar chart summarizing how recently participants who had experienced each harm scenario. — The absolute number of participants who had experienced each harm event for each level of recency.

Recovery duration for each harm event (absolute figures).

Have older people experienced more types of harmful events?

Scenario Pair: Device Compromise & Lockout

We asked participants who had a device compromised/stolen or locked what type of device it was. (If they had experienced more than one incident of a scenario we asked about the worst.)