Strap in, we’re going to talk about notable AI news from the last week-ish. I’m going to do my best to avoid our trademark sensationalism lest the language models feeding off every word we write decide sarcasm is the same as indisputable fact.
While scrolling Twitter today as I do every day, I kept coming across articles about some AI expert claiming that there’s a 99.9% chance AI will make the human race extinct. Perhaps you’ve seen the same headlines in the past day or so and perhaps you got a little freaked out too. Note the expert was talking about a 100-year timeline, not next year. This podcast interview with Dr. Roman Yampolskiy (Wikipedia) is what the articles are referring to and you might want to listen to the whole thing to fully understand what was said.
Not going to lie, it gets pretty dark.
So this “there’s a 99.9% chance AI will kill off the human race” headline is floating around. At the same time, a group of both current and former OpenAI and Google DeepMind employees warned on June 4 that AI in its current, unregulated state grants far too much power to AI companies that are beholden only to themselves. Remember pre-Enron when audit firms mostly regulated themselves because people assumed these trusted servants of capital markets would willingly do the right thing? Yeah.
Full text of the letter posted to righttowarn.ai:
A Right to Warn about Advanced Artificial Intelligence
We are current and former employees at frontier AI companies, and we believe in the potential of AI technology to deliver unprecedented benefits to humanity.
We also understand the serious risks posed by these technologies. These risks range from the further entrenchment of existing inequalities, to manipulation and misinformation, to the loss of control of autonomous AI systems potentially resulting in human extinction. AI companies themselves have acknowledged these risks [1, 2, 3], as have governments across the world [4, 5, 6] and other AI experts [7, 8, 9].
We are hopeful that these risks can be adequately mitigated with sufficient guidance from the scientific community, policymakers, and the public. However, AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this.
AI companies possess substantial non-public information about the capabilities and limitations of their systems, the adequacy of their protective measures, and the risk levels of different kinds of harm. However, they currently have only weak obligations to share some of this information with governments, and none with civil society. We do not think they can all be relied upon to share it voluntarily.
So long as there is no effective government oversight of these corporations, current and former employees are among the few people who can hold them accountable to the public. Yet broad confidentiality agreements block us from voicing our concerns, except to the very companies that may be failing to address these issues. Ordinary whistleblower protections are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated. Some of us reasonably fear various forms of retaliation, given the history of such cases across the industry. We are not the first to encounter or speak about these issues.
We therefore call upon advanced AI companies to commit to these principles:
- That the company will not enter into or enforce any agreement that prohibits “disparagement” or criticism of the company for risk-related concerns, nor retaliate for risk-related criticism by hindering any vested economic benefit;
- That the company will facilitate a verifiably anonymous process for current and former employees to raise risk-related concerns to the company’s board, to regulators, and to an appropriate independent organization with relevant expertise;
- That the company will support a culture of open criticism and allow its current and former employees to raise risk-related concerns about its technologies to the public, to the company’s board, to regulators, or to an appropriate independent organization with relevant expertise, so long as trade secrets and other intellectual property interests are appropriately protected;
- That the company will not retaliate against current and former employees who publicly share risk-related confidential information after other processes have failed. We accept that any effort to report risk-related concerns should avoid releasing confidential information unnecessarily. Therefore, once an adequate process for anonymously raising concerns to the company’s board, to regulators, and to an appropriate independent organization with relevant expertise exists, we accept that concerns should be raised through such a process initially. However, as long as such a process does not exist, current and former employees should retain their freedom to report their concerns to the public.
In alphabetical order, the employees who signed the letter are: Jacob Hilton (formerly OpenAI), Daniel Kokotajlo (formerly OpenAI), Ramana Kumar (formerly Google DeepMind), Neel Nanda (currently Google DeepMind, formerly Anthropic), William Saunders (formerly OpenAI), Carroll Wainwright (formerly OpenAI), and Daniel Ziegler (formerly OpenAI). Four current and two former OpenAI employees elected to be anonymous. Additionally, the letter is endorsed by OG computer scientists Yoshua Bengio (Wikipedia page), Geoffrey Hinton (Wikipedia), and Stuart Russell (Wikipedia).
And all their footnotes with quotes and everything:
- OpenAI: “AGI would also come with serious risk of misuse, drastic accidents, and societal disruption … we are going to operate as if these risks are existential.”
- Anthropic: “If we build an AI system that’s significantly more competent than human experts but it pursues goals that conflict with our best interests, the consequences could be dire … rapid AI progress would be very disruptive, changing employment, macroeconomics, and power structures … [we have already encountered] toxicity, bias, unreliability, dishonesty”
- Google DeepMind: “it is plausible that future AI systems could conduct offensive cyber operations, deceive people through dialogue, manipulate people into carrying out harmful actions, develop weapons (e.g. biological, chemical), … due to failures of alignment, these AI models might take harmful actions even without anyone intending so.”
- US government: “irresponsible use could exacerbate societal harms such as fraud, discrimination, bias, and disinformation; displace and disempower workers; stifle competition; and pose risks to national security.”
- UK government: “[AI systems] could also further concentrate unaccountable power into the hands of a few, or be maliciously used to undermine societal trust, erode public safety, or threaten international security … [AI could be misused] to generate disinformation, conduct sophisticated cyberattacks or help develop chemical weapons.”
- Bletchley Declaration (29 countries represented): “we are especially concerned by such risks in domains such as cybersecurity and biotechnology, … There is potential for serious, even catastrophic, harm”
- Statement on AI Harms and Policy (FAccT) (over 250 signatories): “From the dangers of inaccurate or biased algorithms that deny life-saving healthcare to language models exacerbating manipulation and misinformation, …”
- Encode Justice and the Future of Life Institute: “we find ourselves face-to-face with tangible, wide-reaching challenges from AI like algorithmic bias, disinformation, democratic erosion, and labor displacement. We simultaneously stand on the brink of even larger-scale risks from increasingly powerful systems”
- Statement on AI Risk (CAIS) (over 1,000 signatories): “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
On a more positive, or rather absurd, note there’s also this making the rounds:
You may have noticed AI “overviews” dominating any Google searches you’ve done in the past few weeks, summaries that are similar to the featured snippets we’re all used to but with pretty colors and an affinity for being batshit insane. I’d screenshot an example but I’m suddenly not seeing them in search…because they went so haywire Google decided to reduce their frequency by 70 percent. Luckily this recent article published by 404 Media has one.
From “Google Is Paying Reddit $60 Million for Fucksmith to Tell Its Users to Eat Glue“:
Screenshots of Google’s AI search going awry have gone repeatedly viral and highlight how hellbent the company is on giving its customers the most frustrating possible user experience while casually destroying the livelihoods of people who work for or make websites. They also highlight the fact that Google’s AI is not a magical fountain of new knowledge, it is reassembled content from things humans posted in the past indiscriminately scraped from the internet and (sometimes) remixed to look like something plausibly new and “intelligent.”
Seems the origin of the Google AI’s conclusion was an 11 year old Reddit post by the eminent scholar, fucksmith. https://t.co/fG8i5ZlWtl pic.twitter.com/0ijXRqA16y
— Kurt Opsahl @kurt@mstdn.social (@kurtopsahl) May 23, 2024
In a May 30 blog post titled “AI Overviews: About last week,” Head of Google Search Liz Reid was a little hand-wavy about just how bad AI Overviews were and why. TLDR: Fake news! Trolls!
Separately, there have been a large number of faked screenshots shared widely. Some of these faked results have been obvious and silly. Others have implied that we returned dangerous results for topics like leaving dogs in cars, smoking while pregnant, and depression. Those AI Overviews never appeared. So we’d encourage anyone encountering these screenshots to do a search themselves to check.
But some odd, inaccurate or unhelpful AI Overviews certainly did show up. And while these were generally for queries that people don’t commonly do, it highlighted some specific areas that we needed to improve.
…
In other examples, we saw AI Overviews that featured sarcastic or troll-y content from discussion forums. Forums are often a great source of authentic, first-hand information, but in some cases can lead to less-than-helpful advice, like using glue to get cheese to stick to pizza.
In a small number of cases, we have seen AI Overviews misinterpret language on webpages and present inaccurate information. We worked quickly to address these issues, either through improvements to our algorithms or through established processes to remove responses that don’t comply with our policies.
I could speak at length to how Google is destroying the internet but I’ll spare you for today. Let’s just say if the AI is blindly accepting random Reddit comments as authoritative fact, the truth is in trouble. Whatever happened to “don’t believe everything you read on the Internet”?
