The tech journalists at 404 Media learned that OpenAI is hiring hundreds of contractors to read and review a massive stream of real users’ ChatGPT prompts and responses. “Project Lily” is reportedly a program that asks contractors to score or critique ChatGPT’s answers to improve the chatbot’s quality and behavior. The fact that prompts may sometimes be reviewed by humans should not come as a complete surprise. AI companies also monitor conversations for safety reasons. Anthropic, for example,…
Selecting a model for your security operations center (SOC) and digital forensics and incident response (DFIR) tasks is important, but selecting the best one is more involved than you might think. SOC tasks rely on a combination of model efficacy, analysis time, cost, and consistency of results. Cisco Talos tested 66 model and reasoning combinations across offerings from both Anthropic and OpenAI on a log analysis task to see if we could identify a clear winner. Instead, we found a repeatable…
As AI coding assistants become standard tools in engineering workflows, security teams face a new challenge: how do you maintain visibility into what an AI agent is doing (and why) across your organization? When those agents can execute shell commands, read files, call APIs, and interact with internal systems via MCP connectors, you need real-time observability to support threat detection, incident response, and compliance. This post walks through how Elastic's InfoSec team built a monitoring…