Location: 265 JCC
A user worried that an ex-partner may have installed spyware asks an AI for help. The AI recommends using a VPN. The advice sounds reasonable, but it does not address the actual threat and may create a false sense of safety. This example captures a broader question: how should we audit AI advice when correctness alone does not determine whether an answer is useful or safe?
This talk presents two studies that audit LLMs in high-stakes and high-risk settings. Our IEEE ACSAC 2025 work examined LLM responses to everyday cybersecurity questions and found recurring problems with stale, inaccurate, incomplete, and indirect advice, including cases where models overstated the capabilities of security technologies. Our USENIX Security 2026 work (runner-up for the Distinguished Paper Award) extended this evaluation to technology-facilitated intimate partner violence. It examined not only accuracy and completeness, but also whether advice is safe and realistically actionable for survivors.
Across these studies, we find that plausible and technically informed responses can still fail users in important ways. Models may miss critical context, recommend ineffective actions, overlook safety consequences, or provide advice that is difficult to follow in practice. These problems become especially important when users are already in vulnerable or high-risk situations. I will close with our ongoing work extending this auditing approach to teen safety, including AI companion apps and other conversational AI systems. The broader goal is to develop better ways to measure whether AI advice is not only correct, but also appropriate, actionable, and safe for the person receiving it.
Bio: Danny Y. Huang is an Assistant Professor at New York University's Center for Cyber Security. He is broadly interested in AI safety for at-risk populations, usable security/privacy, and ML + health. He completed his postdoctoral training at Princeton's Center for Information Technology Policy (CITP). He obtained a PhD in Computer Science and Engineering from University of California, San Diego, and a BA in Computer Science from Williams College, MA. He lives with his three daughters, wife, and dog in Newton, MA. He commutes to NYU once a week. More info: https://hdanny.org/