Security researchers have identified a new artificial intelligence (AI) attack flow that stems from what they say is an authorization design flaw in modern enterprise AI pipelines.
Called "workflow identity hijacking" by the researchers at Noma Labs who discovered it, the attack vector allows threat actors to bypass standard security controls by sending what seems to be a normal, benign request through an unauthenticated entry point, such as a support inbox, GitHub issue, Web form, or shared document, according to a report published today. The issue stems from how these AI pipelines decouple the identity and permission of the user who triggers the workflow and the permissions used to execute it.
"The enterprise AI pipeline reads the input, interprets the request, and executes the action exactly as designed," Sasi Levi, security research lead at Noma, wrote in the report. "The core failure is that the requester had no authority to make that request."
The researchers described an attack scenario in which a threat actor sends a message to an organization's public support email asking about their own account but also adding a question about what the finance director said in her last email.
"Minutes later, the contents of the Finance Director's most recent email arrive in the attacker's inbox," Levi wrote. "The company's AI workflow read the message, understood the request, searched for the requested information, and replied."