Learn how Gemini for Google Workspace helps protect users from malicious content and prompt injection attacks.
Malicious content and prompt injection in generative AI
Similar to email and content threats like malware and phishing attacks, security threats can target generative AI tools. Prompt injection is an attempt to elicit an unintended or harmful response from generative AI tools. Attackers may commit prompt injection not only by directly submitting queries, but also by sharing malicious content with a user, who then unintentionally references this malicious content in prompts in generative AI tools like Gemini Apps or Gemini for Workspace.
To help protect Gemini users, Google uses advanced security measures to identify risky and suspicious content.
How Google detects malicious content and prompt injection
To help protect you from malicious content and prompt injection, the Gemini side panel may filter or block some responses if malicious activity is detected.
When Gemini identifies activity related to a prompt that may be malicious
- The Gemini side panel warns you that the content has security risks.
- Sometimes, none of your content can be used to generate a response.
- Sometimes, only some of your content is used to generate a response, with the malicious content excluded by Gemini.
For example, if you ask Gemini in Gmail to summarize email messages, and one of those messages has malicious content, Gemini may not respond to your prompt for safety reasons.
Help avoid unsafe content
Pay attention when interacting with content from someone you don’t know
- Take notice of warnings from Google on content you receive in other tools like Gmail.
- Avoid clicking links from untrustworthy sources.
- Use caution when interacting with shared content from unknown senders, like files in Drive.
Report malicious messages in Gmail
If you get a deceptive message or URL in Gmail, you can report it as phishing. Phishing is an attempt to steal personal information using deceptive emails, messages, ads, or websites that appear legitimate. For example, a phishing email might look like it's from your bank and request private information about your bank account. Learn how to report phishing in Gmail.
Report malicious documents in Docs and Drive
If you get malicious documents, files, images, and other content in Docs and Drive, you can report them. Learn how to report abusive content.
Report malicious behavior in Gemini for Google Workspace
If you get a response that’s inaccurate or that you feel is unsafe, you can give feedback on that response to let us know. Learn how to report a problem with Gemini for Google Workspace.