Transparency in Usable Privacy and Security Research: Scholars’ Perspectives, Practices, and Recommendations

Summary

Transparent reporting of research is a crucial aspect of good scientific practice and contributes to trustworthy science. Transparency helps to understand research processes, assess the validity of research contributions, and facilitates replication of studies and reported results. In the face of reproducibility crises in other fields, the security and privacy (SP) research community in general and the usable privacy and security (UPS) community in particular lack clear standards for transparent research reporting. To gain insights into current research transparency practices and associated challenges and obstacles in the UPS community, we report findings from 24 semi-structured interviews with UPS researchers. We find that researchers value research transparency and already apply several transparency reporting practices. However, an implicit community standard without incentives that outweigh challenges and drawbacks appears to prevent further advances in research transparency. Based on our findings, we conclude with recommendations for transparency practices and guidance for publication venues to better incentivize research transparency (e.g., adapting artifact evaluation to typical UPS artifacts like study materials) and to alleviate constraints that hinder transparency (e.g., removing page limits on appendices). We hope our findings can spur community discussion and effort to improve research quality through more transparent research reporting.