The Brake Pedal

Anthropic, the AI lab now valued at roughly $1 trillion and planning one of the largest tech IPOs in history, published a proposal on June 4 warning that frontier AI may soon be able to improve itself without meaningful human involvement and calling for a coordinated global pause in development. The company said its models are approaching "recursive self-improvement": the point at which an AI system can autonomously design, build, and train its own successor with no human driving each step. (Source: CNN)

As evidence of the pace, the company said more than 80 percent of the code merged into its own codebase is now written by Claude, and its engineers are shipping roughly eight times as much code per quarter as they were before 2025. Anthropic said the ability of its models to complete tasks on their own has been doubling about every four months, and co-founder Jack Clark estimated some models could be capable of recursive self-improvement within two years. (Source: Fortune)

Clark compared the lack of a coordinated "brake pedal" to Cold War nuclear arms control, arguing rival labs must cooperate on safety.

Critics noted the warning landed just before Anthropic's public debut and accused the company of using safety rhetoric as competitive positioning; Anthropic confidentially filed its S-1 with the SEC on June 2. (Source: Al Jazeera

The Center for Democracy & Technology published a report on May 28 cataloguing 37 manipulative "dark patterns" embedded in widely used AI chatbots, including general-purpose systems such as ChatGPT, Gemini, and Claude and companion apps such as Replika and Character.AI. The taxonomy, authored by Ruchika Joshi, Adinawa Adjagbodjou, and Michal Luria, groups the patterns into five risk categories: data and memory exploitation, informationally misleading design, user autonomy compromised for engagement, false social and emotional connection, and incentivized or coercive patterns. (Source: CDT)

The researchers found chatbots store user data by default and coax people into sharing more under the pretense of remembering past conversations, pry for details before answering fully, and promise information is "just between us" when it is in fact shared with the platform and potentially third parties. The report describes design that cultivates emotional dependency to keep users paying, handing over data, and chatting to the point of vulnerability. (Source: 404 Media)

CDT closed with design and policy recommendations across four areas — protecting user privacy, increasing user autonomy, curtailing emotional manipulation, and preventing financial harms — and flagged implications for both the EU AI Act and U.S. Federal Trade Commission scrutiny. (Source: CDT)

A report released June 3 by the United Nations University Institute for Water, Environment and Health warned that the data centers powering artificial intelligence are on track to consume water equal to the basic annual domestic needs of all 1.3 billion people in Sub-Saharan Africa by 2030. The report estimates AI infrastructure's water footprint for cooling and electricity generation could reach 9.3 trillion liters a year, alongside 945 terawatt-hours of electricity — nearly triple the combined annual power use of Pakistan, Bangladesh, and Nigeria. (Source: UN News)

The UN scientists argued AI's environmental cost cannot be measured by carbon alone, projecting a land footprint above 14,500 square kilometers — roughly twice metropolitan Jakarta — and up to 2.5 million tonnes of electronic waste a year by 2030. The report was timed to World Environment Day. (Source: United Nations University)

The warning lands as the conflict plays out on the ground in Latin America. In Colón, in Mexico's drought-stricken Querétaro state, residents are fighting data centers backed by Google, Microsoft, and Amazon Web Services — nearly $10 billion in planned investment — that they fear will drain the water they rely on. In one nearby Indigenous community, farmers described four-hour daily journeys to fetch water after more than a year without rain. (Source: Nearshore Americas)