Making AI trustworthy: Can we overcome black-box hallucinations?

Like most engineers, as a kid I could answer elementary school math problems by just filling in the answers.

But when I didn’t “show my work,” my teachers would dock points; the right answer wasn’t worth much without an explanation. Yet, those lofty standards for explainability in long division somehow don’t seem to apply to AI systems, even those making crucial, life-impacting decisions.

The major AI players that fill today’s headlines and feed stock market frenzies — OpenAI, Google, Microsoft — operate their platforms on black-box models. A query goes in one side and an answer spits out the other side, but we have no idea what data or reasoning the AI used to provide that answer.

Most of these black-box AI platforms are built on a decades-old technology framework called a “neural network.” These AI models are abstract representations of the vast amounts of data on which they are trained; they are not directly connected to training data. Thus, black-box AIs infer and extrapolate based on what they believe to be the most likely answer, not actual data.

Sometimes this complex predictive process spirals out of control and the AI “hallucinates.” By nature, black-box AI is inherently untrustworthy because it cannot be held accountable for its actions. If you can’t see why or how the AI makes a prediction, you have no way of knowing if it used false, compromised, or biased information or algorithms to come to that conclusion.

While neural networks are incredibly powerful and here to stay, there is another under-the-radar AI framework gaining prominence: instance-based learning (IBL). And it’s everything neural networks are not. IBL is AI that users can trust, audit, and explain. IBL traces every single decision back to the training data used to reach that conclusion.

By nature, black-box AI is inherently untrustworthy because it cannot be held accountable for its actions.

IBL can explain every decision because the AI does not generate an abstract model of the data, but instead makes decisions from the data itself. And users can audit AI built on IBL, interrogating it to find out why and how it made decisions, and then intervening to correct mistakes or bias.

This all works because IBL stores training data (“instances”) in memory and, aligned with the principles of “nearest neighbors,” makes predictions about new instances given their physical relationship to existing instances. IBL is data-centric, so individual data points can be directly compared against each other to gain insight into the dataset and the predictions. In other words, IBL “shows its work.”

The potential for such understandable AI is clear. Companies, governments, and any other regulated entities that want to deploy AI in a trustworthy, explainable, and auditable way could use IBL AI to meet regulatory and compliance standards. IBL AI will also be particularly useful for any applications where bias allegations are rampant — hiring, college admissions, legal cases, and so on.

YouTube tests a search feature where users hum to identify songs

YouTube announced a new experiment on Android devices that determines a song via humming—which seems like a major step up from Apple’s music recognition app Shazam.

As noted on YouTube’s support page, the video-sharing platform is testing a search-by-song capability on the Android version of the app that allows users to figure out a song on YouTube by humming, singing or recording a song.

Users who have access to the experiment can toggle from YouTube voice search to the new song search feature and hum, sing or record a song for three or more seconds. The platform then identifies the tune and directs the user to relevant YouTube videos featuring the searched song, whether that be the official music video, user-generated content or Shorts.

The search-by-song capability is only available to a small portion of Android users. If the feature rolls out more widely, we can see it being helpful for many, as YouTube is a popular destination for looking up songs.

YouTube’s latest experiment probably sounds familiar to some users. In 2020, YouTube’s parent company Google first launched the capability on the Google app, Google Search widget and Google Assistant, letting users figure out a song by humming, whistling or singing into the microphone icon. However, the main difference appears to be that Google’s feature requires users to hum for 10-15 seconds in order to identify the song.

As Google previously explained, its feature is built on machine learning models that can match a person’s hum to a song’s “fingerprint” or signature melody. The new YouTube test uses the same technology as the Google feature, the company confirmed to TechCrunch.

Other music recognition apps like SoundHound and MusixMatch can also identify songs by singing or humming the tune, but they aren’t as popular compared to YouTube and Google. (Still, we recommend checking them out as well).

Twilio Segment teams with Amazon SageMaker on new customer predictions product

The customer data platform provides a central place to collect first party information about customers, but simply having a pile of data is not the point. Companies want to put it to work to improve customer experience and more precisely target certain groups, based on this information they have stockpiled.

More and more companies are providing tooling to help build applications on top of the data, and today at the Signal customer and developers conference, Twilio Segment announced the launch of CustomerAI Predictions, a tool to help marketers make predictions about how a certain group of customers, meeting a certain set of criteria, will behave.

“We’ve seen marketers struggle with getting access to great quality data for a long time. More recently, we’ve realized that we can help marketers, not just execute their own hypotheses that they already have, but help them actually predict who are the most valuable types of customers to target with different types of campaigns,” Alex Millet, senior director of product at Twilio Segment told TechCrunch. That’s where CustomerAI Predictions comes in.

“There’s huge value that we can get out of that data that’s already being collected by those customers that’s flowing into the CDP.” For example, CustomerAI Predictions could come up with a group of customers most likely to buy a new product, based on a previous contributing event like a product viewed or a product added to a cart.

Twilio Segment CustomerAI Predictions in action helping marketers predict how a certain group of customers will behave.

Image Credits: Twilio

Segment collects information like clickstream data from a website or mobile app, while Twilio brings the communications data to help companies better understand which customers are most engaged, and which could need a push.

To build CustomerAI Predictions, the company teamed with Amazon SageMaker. “When we were looking at how to quickly bring this first predictions product to market, AWS and SageMaker were one of the leaders in terms of the ML backbone infrastructure that allowed us to build those products,” Millet said.

Millet also indicated that the company has a generative AI-based email tool on the roadmap, so marketers could potentially combine the data from CustomerAI Predictions, and then use the upcoming CustomerAI Generative Email to generate a custom email based on the data set in the predictions product.

Twilio, whose core business involves communications APIs, purchased Segment in October 2020 for $3.2 billion, as part of a strategy to expand into other parts of the marketing stack. The company introduced Flex, a fully programmable contact center product in 2018 and acquired SendGrid, an email API platform in the same year for $2 billion.

CustomerAI Predictions is generally available starting today.

Datasaur lets you build a model automatically from a set of labels

Long before people were talking about ChatGPT and generative AI, companies like Datasaur were dealing with the nuts and bolts of building machine learning models, helping label things to train the model. As AI has taken off, this kind of capability has become even more important.

In order to bring model building to more companies without a data science specialist, Datasaur announced the ability to create a model directly from the label data, putting model creation in reach of a much less technical audience. It also announced a $4 million seed extension that closed last December.

Company founder Ivan Lee says the recent surge in AI interest has been great for the company, and actually plays well into the startup’s strategy. “What Datasaur has always strived to be is the best place to gather the training data that you need to feed into your models, whether they are LLMs, or traditional NER models, sentiment analysis or what have you,” Lee told TechCrunch.

“We are just the best interface for these non-technical users to come in and label that data,” he said.

The rise of LLMs is helping raise awareness in general about how AI can help in a business context, but he says that most companies are still very much in the exploratory stage, and they still need products like Datasaur to build models. Lee says one of his goals from the start has been to democratize AI, particularly around natural language processing, and the new model building feature should put AI in reach of more companies, even those without a specialized expertise.

“And this feature is one I’m particularly excited about because it allows teams without data scientists, without engineers to just markup and label this data however they see fit, and it’ll just automatically train a model for them,” Lee said.

Lee sees this as a way to move beyond the initial target market of data scientists. “Now we’re going to open it up so construction companies, law firms, marketing companies, who may not have a data engineering background, but can still build NLP models [based on their training data].”

He says he has been able to limit the amount of venture investment he has taken – the previous seed was a modest $3.9 million in 2020 – because he operates leanly. His engineering team is mostly in Indonesia, and while he expects to hire, he takes pride in operating the company in an efficient manner.

“My philosophy has always been profitability, grow in a scalable manner, never grow at all costs,” Lee said. That means he considers every hire and the impact it will have on the business.

By having a remote, cross-cultural workforce, employees can learn from each other and that brings a diversity to the company by its nature. “There is a significant difference in the workplace culture between the U.S. and how things operate in Indonesia. And so one thing is we’ve had to be intentional about capturing the best of both worlds,” he said. That could mean encouraging Indonesian colleagues to speak up or push back on what a manager is saying, which is something they are loath to do culturally. “We’ve been very proactive about encouraging that,” he said.

But he says there’s a lot U.S. employees can learn about how things operate in Asia, as well, like respect for your colleagues and this culture of putting the team first, and he has had to help the teams navigate these cultural differences.

The $4 million investment was led by Initialized Capital with participation from HNVR, Gold House Ventures and TenOneTen. The company has raised a total of $7.9 million.

Value sensitive design and AI: A reconsideration

Phil Hopkins, Director of Product Management at Amazon Music delves into the deeper implications of Value Sensitive Design (VSD), a discipline that extends beyond engineering practices. Learn why understanding and incorporating human values are paramount in product design decisions and how VSD offers valuable preparation for this critical moment. Read more »

The post Value sensitive design and AI: A reconsideration appeared first on Mind the Product.

Steg.AI puts deep learning on the job in a clever evolution of watermarking

Watermarking an image to mark is one’s own is something that has value across countless domains, but these days it’s more difficult than just adding a logo in the corner. Steg.AI lets creators embed a nearly invisible watermark using deep learning, defying the usual “resize and resave” countermeasures.

Ownership of digital assets has had a complex few years, what with NFTs and AI generation shaking up what was a fairly low-intensity field before. If you really need to prove the provenance of a piece of media, there have been ways of encoding that data into images or audio, but these tend to be easily defeated by trivial changes like saving the PNG as a JPEG. More robust watermarks tend to be visible or audible, like a plainly visible pattern or code on the image.

An invisible watermark that can easily be applied, just as easily detected, and which is robust against transformation and re-encoding is something many a creator would take advantage of. IP theft, whether intentional or accidental, is rife online and the ability to say “look, I can prove I made this” — or that an AI made it — is increasingly vital.

Steg.AI has been working on a deep learning approach to this problem for years, as evidenced by this 2019 CVPR paper and the receipt of both Phase I and II SBIR government grants. Co-founders (and co-authors) Eric Wengrowski and Kristin Dana worked for years before that in academic research; Dana was Wengrowski’s PhD advisor.

While Wengrowski noted that though they have made numerous advances since 2019, the paper does show the general shape of their approach.

“Imagine a generative AI company creates an image and Steg watermarks it before delivering it to the end user,” he wrote in an email to TechCrunch. “The end user might post the AI-generated image on social media. Copies of the deployed image will still contain the Steg.AI watermark, even if the image is resized, compressed, screenshotted, or has its traditional metadata deleted. Steg.AI watermarks are so robust that they can be scanned from an electronic display or printout using an iPhone camera.”

Although they understandably did not want to provide the exact details of the process, it works more or less like this: instead of having a static watermark that must be awkwardly layered over a piece of media, the company has a matched pair of machine learning models that customize the watermark to the image. The encoding algorithm identifies the best places to modify the image in such a way that people won’t perceive it, but that the decoding algorithm can pick out easily — since it uses the same process, it knows where to look.

The company described it as a bit like an invisible and largely immutable QR code, but would not say how much data can actually be embedded in a piece of media. If it really is anything like a QR code, it can have a kilobyte or three, which doesn’t sound like a lot, but is enough for a URL, hash, and other plaintext data. Multiple-page documents or frames in a video could have unique codes, multiplying this amount. But this is just my speculation.

Steg.AI provided multiple images with watermarks for me to inspect, some of which you can see embedded here. I was also provided (and asked not to share) the matching pre-watermark images; while on close inspection some perturbations were visible, if I didn’t know to look for them I likely would have missed them, or written them off as ordinary JPEG artifacts.

Yes, this one is watermarked.

Here’s another, of Hokusai’s most famous work:

Image Credits: Hokusai / The Art Institute of Chicago

You can imagine how such a subtle mark might be useful for a stock photography provider, a creator posting their images on Instagram, a movie studio distributing pre-release copies of a feature, or a company looking to mark its confidential documents. And these are all use cases Steg.AI is looking at.

It wasn’t a home run from the start. Early on, after talking with potential customers, “we realized that a lot of our initial product ideas were bad,” recalled Wengrowski. But they found that robustness, a key differentiator of their approach, was definitely valuable, and since then have found traction among “companies where there is strong consumer appetite for leaked information,” such as consumer electronics brands.

“We’ve really been surprised by the breath of customers who see deep value in our products,” he wrote. Their approach is to provide enterprise-level SaaS integrations, for instance with a digital asset management platform — that way no one has to say watermark that before sending it out; all media is marked and tracked as part of the normal handling process.

Concept illustration of a Steg.AI app verifying an image.

An image could be traced back to its source, and changes made along the way could conceivably be detected as well. Or alternatively, the app or API could provide a confidence level that the image has not been manipulated — something many an editorial photography manager would appreciate.

This type of thing has the potential to become an industry standard — both because they want it and because it may in the future be required. AI companies just recently agreed to pursue research around watermarking AI content, and something like this would be a useful stopgap while a deeper method of detecting generated media is considered.

Steg.AI has gotten this far with NSF grants and angel investment totaling $1.2 million, but just announced a $5 million A round led by Paladin Capital Group, with participation from Washington Square Angels, the NYU Innovation Venture Fund, and angel investors, Alexander Lavin, Eli Adler, Brian Early and Chen-Ping Yu.

It’s not just Netflix’s $900K AI jobs, it’s the hypocrisy

The discovery of an AI product manager role at Netflix with a pay ceiling of $900,000 sent critics of the company and entertainment industry wild yesterday. That isn’t the only such listing, and likely not even the most lucrative. No one should be surprised that one of the biggest tech companies in the world is paying top dollar for machine learning talent — but that doesn’t mean striking writers and actors shouldn’t call out the hypocrisy on display.

So what are these jobs? In addition to the overall product manager one, there are five other roles with obvious machine learning responsibilities and likely more if you were to scour the requirements and duties of others.

An engineering manager in member satisfaction ML — their recommendation engine, probably — could earn as much as $849K, but the floor for the “market range” is $449K. That’s where the conversation starts! An L6 research scientist in ML could earn $390K to $900K, and the technical director of their ML R&D tech lab would make $450K-650K. There are some L5 software engineer and research scientist positions open for a more modest $100K-$700K.

One comparison that was quickly made is to the average SAG member, who earns less than $30K from acting per year. Superficially, Netflix paying half a million to its AI researchers so that they can obsolete the actors and writers altogether is the kind of Evil Corp move we have all come to expect. But that’s not quite what’s happening here.

While I have no doubt that Netflix is screwing over its talent in numerous ways, just like every other big studio, streaming platform, and production company, it’s important for those on the side of labor to ensure complaints have a sound basis — or they’ll be dismissed from the negotiating table.

The fact is that Netflix is among the largest and most successful tech companies in the world. While it is a novelty to have its name listed in the power acronym FAANG as well as next to megastudios like Disney and Universal, it also means that it must fulfill two sets of responsibilities.

As a tech company, Netflix is, like every other company on Earth, exploring the capabilities of AI. As you may have guessed from the billions of dollars being invested in this sector, it’s full of promise in a lot of ways that aren’t actually connected to the controversial generative models for art, voice, and writing, which for the most part have yet to demonstrate real value.

No doubt they are exploring those things too, but most companies remain extremely skeptical of generative AI for a lot of reasons. If you read the actual job descriptions, you’ll see that none actually pertain to content creation:

-You will lead requirements, design, and implementation of Metaflow product improvements…

-You will lead a team of experts in these techniques to understand how members experience titles, and how that changes their long-term assessment of their satisfaction with the Netflix service.

-…incubate and prototype concepts with the intent to eventually build a complete team to ship something new that could change the games industry and reach player audiences in new ways, as well as influencing adoption of AI technologies and tooling that are likely to level up our practices.

-…we are venturing further into exciting new innovations in personalization, discovery, experimentation, backend operations, and more, all driven by research at the frontiers of ML

-…Collect feedback and understand user needs from ML/AI practitioners and application engineers across Netflix, deriving product requirements and sizing their importance to then prioritize areas of investment.

-We are looking for an Applied Machine Learning Scientist to develop algorithms that power high quality localization at scale…

Sure, the last one is likely generative dubbing, or perhaps improved subtitle translation. And this doesn’t mean Netflix isn’t working on generative stuff too. But these are the jobs we’re actually seeing advertised, and most are generic “we want to see what we can do with AI to make stuff better and more efficient.”

AI applies across countless domains, as we chronicle in our regular roundup of research. A couple weeks ago it helped find new Nasca lines! But it’s also used in image processing, noise reduction, motion capture, network traffic flow, and datacenter power monitoring, all of which are relevant to a company like Netflix. Any company of this size that is not investing hundreds of millions in AI research is going to be left behind. If Disney or Max develops a compression algorithm that halves the bandwidth needed for good 4K video, or cracks the recommendation code, that’s a huge advantage.

So, why am I out here defending a giant corporation that clearly should be paying its writers and actors more?

Because if the unions and their supporters are going to take Netflix to task, as they should given the deplorable state of residuals and IP ownership, they can’t base their outrage on industry standard practices that are necessary for a tech company to succeed in the current era.

We don’t have to like that AI researchers are being paid half a million while an actress from a hit show a couple years back gets a check for $35. But this portion of Netflix’s inequity is, honestly, out of their control. They’re doing what is required of them there. Ask around: anyone with serious experience in machine learning and running an outfit is among the most sought-after people in the world right now. Their salaries are grossly inflated, yes — they’re the A-listers of tech right now, and this is their moment.

The problem is that by Netflix demonstrating its ability to do what’s needed in one area, they call attention to their failure to do what is required of them elsewhere — namely in support of creators, whose entire relationship with distribution platforms needs to be rewritten from scratch.

The threat of creators jumping ship and going to another streaming platform is very real. The next big indie horror hit is probably already working with A24 instead of one of the big guys because A24 gave the union everything they asked for. That’s 50 million dollars in the debit column because someone didn’t come to the table.

By all means let’s get up in arms about inequity — but if this anger is to take effect, it needs to be grounded in reality and targeted properly. Hiring an AI researcher for an extravagant salary to refine their recommendation engine isn’t the problem on its own — it’s the hypocrisy demonstrated by Netflix (and every other company doing this, probably all of them) showing that it is willing to pay some people what they’re worth, and other people as little as they can get away with. That’s a deliberate choice, and one that the striking creators hopefully can ensure is no longer possible in the future.

Graft is building an AI development platform for the masses

When Graft launched its AI development platform in beta last year, it was looking to build something that put AI within reach of every company, not just the largest ones with tons of engineering resources.

Today, the company announced a $10 million seed investment. While it was at it, the startup announced that it’s inviting companies to request access to the platform as it begins to open up the offering to a larger number of companies in a controlled fashion.

Graft co-founder and CEO Adam Oliner came up with the idea for the company while he was running AI at Slack. Today, he sees how the growing excitement around ChatGPT has brought AI to the forefront of business conversations, but he says that there’s a big difference between playing with ChatGPT and building a production-ready AI application.

“The shiny toys just aren’t built for production, and the established platforms are largely unusable by non-experts. So we see Graft as filling in that gap in the market with a production grade, modern AI platform that’s built for everyone,” Oliner told TechCrunch.

As Oliner sees it, ChatGPT has served to show what’s possible, even if developing AI applications hasn’t necessarily gotten any easier. “While you can abstract away a ton of that complexity using things like large language models, the path to production has not gotten any simpler, and in some ways has gotten more complex,” he said. “These models are complex and not readily explainable. They’re large and unwieldy. There are new questions related to compliance, privacy, AI ethics and so on that weren’t present before.”

To make it easier, the company has introduced a handful of apps to help customers get going quickly without having to build something themselves from scratch. “We’re introducing this notion of apps, which are templatized use cases that you can quickly and easily instantiate into full-blown production use cases based on your data,” he said.

Among their current offerings are visual search or finding customer champions. He says they have tried to make it simple to get started — you just need to create a Graft account, choose one of the pre-defined templates and point to your data. Graft handles the infrastructure to run the application for you.

Today’s $10 million investment was led by Radical Ventures, with participation from GV. The company has now raised a total of $14.5 million, having raised a $4.5 million pre-seed round last year.

AI leaders warn Senate of twin risks: moving too slow and moving too fast

Leaders from the AI research world appeared before the Senate Judiciary Committee to discuss and answer questions about the nascent technology. Their broadly unanimous opinions generally fell into two categories: we need to act soon, but with a light touch — risking AI abuse if we don’t move forward, or a hamstrung industry if we rush it.

The panel of experts at today’s hearing included Anthropic co-founder Dario Amodei, UC Berkeley’s Stuart Russell, and longtime AI researcher Yoshua Bengio.

The two-hour hearing was largely free of the acrimony and grandstanding one sees more often in House hearings, though not entirely so. You can watch the whole thing here, but I’ve distilled each speaker’s main points below.

Dario Amodei

What can we do now? (Each expert was first asked what they think are the most important short-term steps.)

1. Secure the supply chain. There are bottlenecks and vulnerabilities in the hardware we rely on to research and provide AI, and some are at risk due to geopolitical factors (e.g. TSMC in Taiwan) and IP or safety issues.

2. Create a testing and auditing process like what we have for vehicles and electronics. And develop a “rigorous battery of safety tests.” He noted, however, that the science for establishing these things is “in its infancy.” Risks and dangers must be defined in order to develop standards, and those standards need strong enforcement.

He compared the AI industry now to airplanes a few years after the Wright brothers flew. There is an obvious need for regulation, but it needs to be a living, adaptive regulator that can respond to new developments.

Of the immediate risks, he highlighted misinformation, deepfakes, and propaganda during an election season as being most worrisome.

Amodei managed not to bite at Sen. Josh Hawley’s (R-MO) bait regarding Google investing in Anthropic and how adding Anthropic’s models to Google’s attention business could be disastrous. Amodei demurred, perhaps allowing the obvious fact that Google is developing its own such models speak for itself.

Yoshua Bengio

What can we do now?

1. Limit who has access to large-scale AI models and create incentives for security and safety.

2. Alignment: Ensure models act as intended.

3. Track raw power and who has access to the scale of hardware needed to produce these models.

Bengio repeatedly emphasized the need to fund AI safety research at a global scale. We don’t really know what we’re doing, he said, and in order to perform things like independent audits of AI capabilities and alignment, we need not just more knowledge but extensive cooperation (rather than competition) between nations.

He suggested that social media accounts should be “restricted to actual human beings that have identified themselves, ideally in person.” This is in all likelihood a total non-starter, for reasons we’ve observed for many years.

Though right now there is a focus on larger, well-resourced organizations, he pointed out that pre-trained large models can easily be fine-tuned. Bad actors don’t need a giant datacenter or really even a lot of expertise to cause real damage.

In his closing remarks, he said that the U.S. and other countries need to focus on creating a single regulatory entity each in order to better coordinate and avoid bureaucratic slowdown.

Stuart Russell

What can we do now?

1. Create an absolute right to know if one is interacting with a person or a machine.

2. Outlaw algorithms that can decide to kill human beings, at any scale.

3. Mandate a kill switch if AI systems break into other computers or replicate themselves.

4. Require systems that break rules to be withdrawn from the market, like an involuntary recall.

His idea of the most pressing risk is “external impact campaigns” using personalized AI. As he put it:

We can present to the system a great deal of information about an individual, everything they’ve ever written or published on Twitter or Facebook… train the system, and ask it to generate a disinformation campaign particularly for that person. And we can do that for a million people before lunch. That has a far greater effect than spamming and broadcasting of false info that is not tailored to the individual.

Russell and the others agreed that while there is lots of interesting activity around labeling, watermarking, and detecting AI, these efforts are fragmented and rudimentary. In other words, don’t expect much — and certainly not in time for the election, which the Committee was asking about.

He pointed out that the amount of money going to AI startups is on the order of ten billion per month, though he did not cite his source on this number. Professor Russell is well-informed but seems to have a penchant for eye-popping numbers, like AI’s “cash value of at least 14 quadrillion dollars.” At any rate even a few billion per month would put it well beyond what the U.S. spends on a dozen fields of basic research through the National Science Foundations, let alone AI safety. Open up the purse strings, he all but said.

Asked about China, he noted that the country’s expertise generally in AI has been “slightly overstated” and that “they have a pretty good academic sector that they’re in the process of ruining.” Their copycat LLMs are no threat to the likes of OpenAI and Anthropic, but China is predictably well ahead in terms of surveillance, such as voice and gait identification.

In their concluding remarks of what steps should be taken first, all three pointed to, essentially, investing in basic research so that the necessary testing, auditing, and enforcement schemes proposed will be based on rigorous science and not outdated or industry-suggested ideas.

Sen. Blumenthal (D-CT) responded that this hearing was intended to help inform the creation of a government body that can move quickly, “because we have no time to waste.”

“I don’t know who the Prometheus is on AI,” he said, “but I know we have a lot of work to make that the fire here is used productively.”

And presumably also to make sure said Prometheus doesn’t end up on a mountainside with feds picking at his liver.

Top AI companies visit the White House to make ‘voluntary’ safety commitments

While substantive AI legislation may still be years away, the industry is moving at light speed and many — including the White House — are worried that it may get carried away. So the Biden administration has collected “voluntary commitments” from 7 of the biggest AI developers to pursue shared safety and transparency goals ahead of a planned Executive Order.

OpenAI, Anthropic, Google, Inflection, Microsoft, Meta, and Amazon are the companies taking part in this non-binding agreement, and will send representatives to the White House to meet with President Biden today.

To be clear, there is no rule or enforcement being proposed here — the practices agreed to are purely voluntary. But although no government agency will hold a company accountable if it shirks a few, it will also likely be a matter of public record.

Here’s the list of attendees at the White House gig:

  • Brad Smith, President, Microsoft
  • Kent Walker, President, Google
  • Dario Amodei, CEO, Anthropic
  • Mustafa Suleyman, CEO, Inflection AI
  • Nick Clegg, President, Meta
  • Greg Brockman, President, OpenAI
  • Adam Selipsky, CEO, Amazon Web Services

No underlings, but no billionaires, either. (And no women.)

The seven companies (and likely others that didn’t get the red carpet treatment but will want to ride along) have committed to the following:

  • Internal and external security tests of AI systems before release, including adversarial “red teaming” by experts outside the company.
  • Share information across government, academia, and “civil society” on AI risks and mitigation techniques (such as preventing “jailbreaking”).
  • Invest in cybersecurity and “insider threat safeguards” to protect private model data like weights. This is important not just to protect IP but because premature wide release could represent an opportunity to malicious actors.
  • Facilitate third-party discovery and reporting of vulnerabilities, e.g. a bug bounty program or domain expert analysis.
  • Develop robust watermarking or some other way of marking AI-generated content.
  • Report AI systems’ “capabilities, limitations, and areas of appropriate and inappropriate use.” Good luck getting a straight answer on this one.
  • Prioritize research on societal risks like systematic bias or privacy issues.
  • Develop and deploy AI “to help address society’s greatest challenges” like cancer prevention and climate change. (Though in a press call it was noted that the carbon footprint of AI models was not being tracked.)

Though the above are voluntary, one can easily imagine that the threat of an Executive Order — they are “currently developing” one — is there to encourage compliance. For instance, if some companies fail to allow external security testing of their models before release, the E.O. may develop a paragraph directing the FTC to look closely at AI products claiming robust security. (One E.O. is already in force asking agencies to watch out for bias in development and use of AI.)

The White House is plainly eager to get out ahead of this next big wave of tech, having been caught somewhat flat-footed by the disruptive capabilities of social media. The President and Vice President have both met with industry leaders and solicited advice on a national AI strategy, as well is dedicating a good deal of funding to new AI research centers and programs. Of course the national science and research apparatus is well ahead of them, as this highly comprehensive (though necessarily slightly out of date) research challenges and opportunities report from the DOE and National Labs shows.