Ask Gemini in Chrome to complete tasks for you with auto browse

With Chrome auto browse, you can ask Gemini Spark to complete tasks that involve browsing the web, like shopping, booking travel, or making dinner reservations. To work on your tasks, Spark uses your info from sources like Connected Apps, skills, chats, tasks, websites you’re logged into, Personal Intelligence, location, and more. 

Gemini Spark can browse the web in multiple ways, one of which is by connecting to your own device (aka your local Chrome browser). Currently, only Gemini Spark in desktop Chrome can use auto browse. It might also use a remote browser (which is a separate, new browser instance, not your local browser in the cloud). 

If you use your local browser: Gemini Spark has access to all the same sites that you do, including sites you’re signed into. When acting on your behalf, Gemini Spark will share necessary info with third parties. This could include your name, contact information, files, preferences, and info you might find sensitive. With your permission, it can also use login info you saved in Password Manager to sign into loyalty programs and online accounts. 

To use Spark via your local browser, you’ll need to keep your device and Google Chrome awake and running. If you later close or turn off your device before the task is finished, or the device isn’t available, Spark might use a remote browser to complete the task.

If you use the remote browser: Your task can continue even if you close your device. Note that at some stages -- like when a website asks you to sign in -- the task will stop until you take over and do it yourself.

Notes:

• The first time Gemini Spark needs to browse the web for you on this device, Chrome will ask you for permission to connect Chrome to Spark.
• After that, for every task that involves web browsing, Spark will ask for confirmation first.
• You can turn off Gemini Spark’s ability to autobrowse on a device. Click More   Settings AI Innovations Gemini in Chrome.

Keep track of tasks

If Gemini Spark is busy using auto browse for a task, you’ll see it in your Recent Tasks. At any time, you can stop Gemini from web browsing. In the Gemini in Chrome side panel, click Stop [icon]. If you do, Gemini tries to figure out the next steps to continue your task. This may involve use of the remote browser to web browse or activation of a different tool. Or it could mean that Gemini abandons the task. 

If you want to completely cancel a task yourself, in the Gemini Spark text box, click Stop . 

Depending on the browser you used, your browsing data can be stored in two different places: your local browser and the remote browser (or virtual machine). 

• Learn more about your data use in the remote browser. 
• Learn how to delete remote browser data. 
• Learn how to delete your local browser data in Chrome.

AI agents, like auto browse in Gemini in Chrome, are helpful and powerful tools, but it’s important to be aware of possible risks. Remember that Gemini in Chrome can make mistakes and do unexpected things.

About prompt injection

Prompt injection is an attempt to elicit an unintended or harmful response from generative AI tools like Gemini in Chrome. This happens when content, like a website, an email, a document, or multimedia, has malicious instructions that might be hidden from you but visible to the AI agent.

If an AI agent reads these instructions while doing a task, it might be tricked into doing unintended things. For example, a malicious instruction could tell the agent to:

• Take your private info from your emails or documents and post it on a public website
• Send your emails in Gmail to an external service without you knowing
• Expose insights about you based on your data in connected apps

Gemini in Chrome can share your info with websites

Auto-browse works across your open tabs to complete actions and get things done on your behalf. Gemini can choose sites to visit to fulfill your request. Gemini may use your personal information (such as data from Connected Apps like Google Workspace) to complete these tasks more accurately. When completing an auto browse task on a website, Gemini may share your information with that site.

Note: You can manage your data-sharing preferences and Connected Apps in your Gemini settings.

Gemini has access to your local browsing state

Auto browse has access to the same sites that you do, including sites you’re signed into.

Gemini in Chrome can make mistakes

Gemini in Chrome might misunderstand what you want it to do or the website it’s browsing. This could lead to unintended actions, like:

• Following instructions incorrectly or clicking the wrong link or button
• Completing a purchase or ordering something without your permission
• Adding the wrong quantity of an item to your cart or forgetting to add an item
• Confirming a task is done when it’s not or could have done more
• Asking you to take over when it can normally do the step

Learn about how we address these risks in Gemini in Chrome.

Chrome has safeguards for auto browse that aim to reduce the possible risks of AI agents, including unintended or potentially harmful actions. While these do not guarantee protection against all risks, these safeguards include:

• “Take over task”: Gemini in Chrome may ask you to take over the task to complete specific actions, like:
  • Finalizing financial transactions
  • Accepting terms of service
  • Creating an account
• Requests for user confirmation: Gemini in Chrome aims to ask for your review and confirmation before certain actions, like:
  • Starting a task
  • Sending communications
  • Modifying your data
  • Submitting web forms
  • Scheduling events
  • Accessing sites with highly sensitive financial or health data
• Requests for permission: Gemini in Chrome asks for your permission before it gets help from Google Password Manager to sign in to sites for you. Google Password Manager does not share your passwords with Gemini in Chrome. You can remove this permission at any time.
• Prohibited task recognition: Gemini in Chrome has safeguards designed to help it recognize and not act on certain types of requests that could be harmful or are outside of its intended purpose, including tasks that may violate the Generative AI Prohibited Use Policy.
• Site & action restrictions: Gemini in Chrome restricts its activity to the websites and actions relevant to your task. This helps protect your data on other unrelated sites and helps prevent actions that don't align with your task. These are just some ways we protect against prompt injection.

Learn more about the safeguards in Chrome for auto browse.

Chrome’s safeguards for auto browse described above don’t guarantee protection against all risks.

Monitor important & sensitive tasks

Monitoring your tasks is the most important way to protect against risk while using auto browse. Review confirmation requests and clarifying questions closely. Use auto browse carefully and safely, and only give Gemini in Chrome tasks you’re comfortable with. Be extra watchful when Gemini in Chrome:

• Handles an important task
• Is signed in to websites with sensitive info, like financial, legal, and medical info
• Does anything where a mistake could be a problem
• Uses Connected apps, like Google Workspace, in the same chat

You’re in control

If Gemini in Chrome is doing something you don't like, you can stop it by clicking Stop in Gemini in Chrome or Take over task on the tab that Gemini in Chrome is using. Gemini in Chrome’s safeguards aren’t intended to replace your monitoring.