Beware of Prompt Injection Risk When Summarizing Emails with Gemini

Attackers can hide malicious instructions inside emails to trick Google's Gemini large language model into delivering deceptive messages to end users. The vulnerability could be abused to trick users into taking actions that could lead to sensitive information being stolen. Google has introduced new defenses against such attacks, including sanitizing markdown and detecting malicious prompts using machine learning models.

One challenge posed by this vulnerability is that it can be targeted via low-tech means without needing email attachments or links. To defend against such exploits, tools need to strip or neutralize certain inline styles and users should be trained to understand that Gemini summaries are informational, not authoritative security alerts.

This vulnerability alert comes as more service providers are adding automated services driven by large language models to their products. Google's Gemini, introduced last year, assists users in summarizing email threads and other content within its apps.

According to the source: BankInfoSecurity.