Attackers are poisoning popular AI chatbots with misinformation and phishing links so they serve up this content to users as facts or trusted websites. The content-manipulation campaign is targeting popular brands across various industries, creating reputational damage and other ramifications for the organizations.
Researchers from Vigilance Security identified the campaign, dubbed "Dark Sourcery," which essentially uses social engineering on OpenAI's ChatGPT, Google Gemini, and Google AI Overview, according to research published today.
"Attackers are flooding the web with carefully optimized posts, PDFs, reviews, and fake support pages, to trick AI into presenting fraudulent phone numbers, email addresses, and login pages," Ariel Simon, vice president of research at Vigilance, wrote in the post.
Vigilance researchers identified tens of thousands of malicious pages containing fake support numbers, email addresses, login pages, software-update information, and other fraudulent details as part of the attackers' aim to manipulate the content delivered by AI chatbots, he said.