With a team of researchers from Princeton University and UC Berkeley, we conducted a mixed methods study on how system administrators manage software updates for their organization.
Duration: 10 months
Team Size: 5
HCI Methods Employed
Mixed methods study design, user interviews, ethnographic research, survey design, qualitative and quantitative data analysis
You can find the talk done by my co-author at the SOUP 2019 conference here.
Abstract
Software updates play a central role in enabling hosts to counter newly discovered security vulnerabilities or emergent security concerns.
However, many security breaches occur because software updates are not installed in a timely manner—or at all. While recent studies have investigated the updating practices of end users, system administrators have received less attention.
This population is of particular importance as system administrators manage numerous and often diverse machines for organizations or networks,
and insecurities at these hosts can lead to crippling attacks that inflict financial damages, leak private data, or tarnish reputations.
In this paper, we study how system administrators manage software updates across multiple hosts in their organizations.
We conducted a large-scale survey of 102 system administrators and interviewed 17 administrators in a semi-structured fashion to understand their processes for managing updates, what works well, what challenges they face, and how the updating process could be improved.
We have three main findings: first, we find that obtaining information—such as discovering available and relevant updates— and acting upon this information remains a challenge throughout the update process.
Second, due to the risk of problematic updates, the vast majority of administrators perform update testing and seek to minimize update disruptions during deployment, difficult tasks that often delay full patch deployment, resulting in expanded vulnerability windows. Third, organizational policies and management decisions can aid or hinder system administrators’ update processes. Moving forward, we propose a variety of directions for improving the update process for system administrators, such as centralizing update information, leveraging administrator notifications, disentangling patches of different types, and shifting organizational cultures.
Paper Accepted by SOUPS 2019
Lessons Learned
At least for this population, the opportunity to talk passionately about their work with an engaged listener with their confidentiality protected was far more motivating than the $20 Amazon gift-card. In fact even the LEGOs we used for recruiting at the LISA conference were a far greater motivator than either the amazon gift-card or entry to win a Samsung Galaxy 8. This research could have been conducted on a lower budget than we did if marketed differently. Even so it was effective with over 100 survey participants and 20 interview participants.
With the guidance of Dr. Chetty, I was able to refine my interview and survey creation skills. I also gained experience using ethnographic techniques to recruit a “difficult to recruit on an academic budget” demographic.
I contributed to this project by helping to designing the interview protocol and survey, recruiting participants, conducting interviews, analyzing the transcripts, and collaborating to write a paper for conference submission.
For this study I immersed myself in system administrator culture by participating on forums and Slack channels frequented by this demographic, as well as attending the LISA conference in San Francisco. It was a great exercise in ethnographic research to understand the nuances of challenges faced by this population in their work.
Results and SOUPs paper submission to be added shortly.
Lisa Marie Rogers 