Google is making it easier for AI agents to scrape websites
Google is building a way for AI agents to interact with websites directly through Chrome, without scraping pages or burning through tokens guessing at the DOM.
Google just launched an early preview of WebMCP, a new browser standard that lets websites expose structured tools directly to AI agents like OpenClaw or Hermes operating inside Chrome.
MCP, or Model Context Protocol, is an open standard that defines how AI agents connect to and interact with external tools and services. WebMCP brings that same idea into the browser layer.
The way agents currently interact with websites is slow and unreliable. They read raw page structure, guess at what’s clickable, and burn through tokens navigating interfaces built for humans. WebMCP gives websites a way to sidestep all of that by defining exactly what an agent can do on the page.
There are two ways sites can expose those capabilities:
- Declarative API: Standard actions defined directly in HTML forms, covering straightforward interactions like filling out fields or submitting requests
- Imperative API: More complex, dynamic interactions that require JavaScript execution for multi-step or conditional flows
Among the things it can handle: booking flows, e-commerce checkouts, customer support ticket creation, and search and filter navigation. The point is that the site defines the action, and the agent executes it cleanly without interpreting the DOM.
The preview is open to developers who sign up for Google’s early preview program. Participants get access to documentation, demos, and the APIs for prototyping.
That framing raises an uncomfortable question. For years, websites have fought unauthorized scraping through rate limiting, CAPTCHAs, robots.txt restrictions, and legal action. WebMCP is essentially a formalized version of the same access those sites spent years blocking. The difference is that it’s opt-in and structured, but the end result, an automated system reading and acting on your site without a human present, is not categorically different.
The bigger unresolved issue is what this does to traffic and monetization. Most ad-based business models depend on humans loading pages, seeing ads, and clicking around. An agent that completes a booking or a checkout through a structured API call may never load the page at all.
Google hasn’t addressed what that looks like for publishers or e-commerce sites that depend on that traffic. WebMCP solves a real problem for agent developers. What it creates for everyone else is less clear.
Bottom line: AI agents are going to interact with websites whether sites define the interface or not. WebMCP gives them a cleaner way to do it, but the broader questions around traffic, ad revenue, and what an agent-first web means for site owners are still completely open.
Source: Chrome for Developers
If you need on-demand GPUs for training, fine-tuning, inference, or running open-source models, give RunPod a try.
- Available hardware: H100, H200, A100, L40S, RTX 4090, RTX 5090, and 30+ more
- Cost: significantly cheaper than AWS or GCP, billed per second, no contracts
- Setup: spins up in under a minute, 30+ regions worldwide
Get the core business tech news delivered straight to your inbox. We track AI, automation, SaaS, and cybersecurity so you don't have to.
Just read what you want, and be done with it.
