Major artificial intelligence developers have agreed to adopt a unified standard for website content scraping. OpenAI, Google, Microsoft, and Anthropic will now recognize a single "policy token" within a website's robots.txt file, allowing publishers to opt out of AI training data collection across all participating companies simultaneously. This move aims to simplify the current fragmented system where website owners must individually configure rules for each AI company's web crawlers. The proposal, developed by Kevin Kelley, a technical lead for standards at Anthropic, was submitted through the Internet Engineering Task Force (IETF). Anthropic has already begun implementing the standard, while OpenAI plans to do so in the coming months. Google and Microsoft are targeting early 2027 for full implementation.
This new standard is designed to streamline the process for website operators wishing to control how their content is used for AI model training. Previously, publishers had to manage separate directives for each AI company's specific crawlers, a task that became increasingly burdensome as the number of AI developers grew. The shared token, proposed as part of an IETF Internet-Draft, allows for a single entry in the robots.txt file to communicate a preference against AI training. This system is intended to work alongside existing web crawling practices, such as those used for search engine indexing, and does not prevent general web crawling. Microsoft has also indicated plans to introduce granular controls for AI training and "AI grounding," a process that directs models to verified sources to reduce the generation of inaccurate information. While Meta has expressed support for a common signal, it has not yet committed to its implementation.
In parallel, Google has achieved a milestone with its Project Suncatcher initiative, successfully running an AI model on a satellite-based Tensor Processing Unit (TPU) in low Earth orbit. The prototype satellite, launched on October 1, 2026, transmitted its first AI-generated response: a haiku composed by the Gemma 4 model. The poem, which reads "Blue marble below / Data streams in silent grace / Starlight feeds my core," was received during a ground-station pass over Svalbard, Norway. This achievement represents a significant step in Google's research into operating data-center-class TPUs in the challenging environment of space. The project aims to gather data on how these specialized chips perform under spaceflight conditions, including radiation and thermal extremes. Engineers are continuing to commission the spacecraft and conduct further machine-learning tests, with future phases focusing on sustained workloads and the long-term reliability of the TPUs in orbit.