Learn how Gemini Apps help protect users from malicious content and prompt injection attacks.
Malicious content & prompt injection in generative AI
Similar to email and content threats like malware and phishing attacks, security threats can target generative AI tools. Prompt injection is an attempt to elicit an unintended or harmful response from generative AI tools. Attackers may commit prompt injection not only by directly submitting queries, but also by sharing malicious content with a user, who then unintentionally references this malicious content in prompts in generative AI tools like Gemini Apps or Gemini for Workspace.
To help protect Gemini users, Google uses advanced security measures to identify risky and suspicious content.
How Google helps detect malicious content & prompt injection
To help protect you from malicious content and prompt injection, Gemini Apps may block some inputs if suspicious activity is detected.
When Gemini identifies activity related to a prompt that may be suspicious
- Gemini Apps provides a warning notification that the content has security risks
- Sometimes none of the content can be used to generate a response
- Sometimes only some of the content is used to generate a response, with the suspicious content excluded by Gemini
For example, if you ask Gemini Apps to summarize email messages, and one of those messages has malicious content, Gemini may not respond to your prompt for safety reasons.
Help avoid unsafe content
Pay attention when interacting with content from someone you don’t know
- Take notice of warnings from Google on content you receive in other tools like Gmail
- Avoid clicking links from untrustworthy sources
- Use caution when interacting with shared content, like public Gemini Apps chats or Gemini Canvas web apps, from unknown providers
Malicious content and prompt injections in generative AI
If you get a response that’s inaccurate or that you feel is unsafe, you can give feedback on that response to let us know.