When Scammers Talk Back:
Understanding Potentially Unwanted Calls via LLM-Based Interaction

Summary

Despite stricter regulations being gradually enforced in telecommunications, potentially unwanted calls (PUCs), including telemarketing and vishing, remain a persistent and evolving threat. However, understanding how PUCs operate in practice has been limited by the lack of real-world datasets with rich conversational content, hindering the development of effective defenses. To fill this gap, we developed an LLM-based phone call honeypot for collecting real-world PUCs, to our knowledge among the first systems to collect such data in the wild. Based on the development, we conducted thematic analysis on 115 hours of conversations collected over 20 months, where we understand how PUCs operate in practice by studying the conversational structure, content, and campaign operations. Our analysis reveals that PUC conversations are highly structured across diverse call topics. We identify a widespread early-call pattern, which we term the hook phase, that callers use to engage and prescreen targets before eliciting sensitive information. Within this phase, we identify an emerging entity, call-to-interaction robocalls. Unlike call-to-action robocalls, which rely on prerecorded audio, call-to-interaction robocalls sustain engagement through interactive questioning, more closely resembling human agents. Beyond the hook phase, we find that malicious callers progressively escalate the sensitivity of personal information over the course of conversations while using persuasion techniques to establish authority and rapport. Our longitudinal analysis further identifies eight major topics that remain present throughout the nearly two-year study period (e.g., Medicare), along with major campaigns that stay active over time, suggesting that PUC operations are durable and underscoring the need for stronger interventions against them.