Klaus secures fresh capital to automatically categorize and score customer interactions

Martin Kõiva was at Pipedrive, leading the company’s customer support organization, when he says he came to the realization that the best way to prevent bad customer interactions is to analyze previous ones, give agents regular check-ins and not rely too strictly on customer feedback. But Kõiva was hampered in his efforts to implement these practices at scale because the tools to do so didn’t exist, he says.

Seeking to build them himself, Kõiva teamed up with Kair Käsper (also ex-Pipedrive) and Egon Sale to co-found Klaus, a customer support product that integrates with clients’ customer relationship management platforms (e.g., Zendesk, Salesforce Service Cloud) to automatically review customer support conversations from channels like web chats. Klaus today closed a €12 million (~$11.49 million) Series A equity round led by Acton Capital, which Kõiva says will be used to support the development and further expansion of Klaus’s software.

For large companies that have millions of support tickets, it is crucial that managers are able to find the conversations that have a meaningful impact on performance. It’s a needle in a haystack,” Kõiva told TechCrunch in an email interview. “Klaus is able to automatically analyze the entire customer support volume and pinpoint which conversations require attention.”

Drawing on customer support tickets, input from managers reviewing agent conversations and customer satisfaction feedback, Klaus trains AI algorithms to perform tasks like automatically categorizing comments from customers and sorting conversations by attributes like complexity. Klaus can perform sentiment analysis in a number of languages out of the box, Kõiva claims, a capability the platform uses to score the “quality” of customer-agent conversations. 


Image Credits: Klaus

“Klaus [can] piece together what ‘good’ and ‘bad’ looks like for each individual customer and, with the help of data science, deliver actionable insights that improve customer service for companies that have millions of support tickets every month,” Kõiva said. “Klaus technology is currently analyzing two million customer conversations every day.”

Automated scoring systems, particularly those that rely on potentially biased sentiment analysis techniques, raise questions about whether customer agents might be evaluated inaccurately or unfairly. When asked about factors like bias, Kõiva said that Klaus takes mitigating steps like removing color-, region-, and gender-specific emojis in the customer feedback data that its algorithms analyze. 

Klaus competes with companies such as MaestroQA, Playvox and Stella Connect. Beyond those, there’s ScopeAI, acquired by Observe.AI in 2021 for its technology that helps companies analyze customer feedback, and Zendesk-owned Cleverly, which automatically tags incoming customer service requests to help categorize the workflow.

Kõiva believes Klaus is well-positioned, however, with a customer base totaling “hundreds” of companies, including Epic Games, SoundCloud and WordPress.com. To continue to stand out, Klaus recently added customer satisfaction survey functionality with automatic tagging, allowing admins to spot trends that they might otherwise miss.  

Klaus has … seen an uptick in interest from companies that are looking to optimize their customer service operations,” Kõiva continued. “Large enterprises also tend to use more outsourced customer service to keep costs flexible during uncertain [economic] times, and Klaus provides a degree of confidence that the quality of the outsourced service is under control.”

Klaus currently employs around 60 people, a number Kõiva expects will grow to over 100 within the next six months. To date, the startup has raised more than $19 million in venture capital.

Klaus secures fresh capital to automatically categorize and score customer interactions by Kyle Wiggers originally published on TechCrunch

Lunio raises $15M to combat click fraud with algorithms

The digital ads market is robust, with Statista predicting that worldwide spend will reach nearly $900 billion by 2026. But fake and fraudulent ad traffic remains a major problem in the space. Global losses from ad fraud totaled $35 billion in 2020 alone. And beyond the wasted spend, invalid traffic can inflate metrics, leading brands to misidentify — and misunderstand — customer segments.

To combat ad fraud, Neil Andrew and Alex Winston co-founded Lunio, which attempts to exclude fake web traffic arriving from different channels by analyzing behavior patterns. The startup today announced that it closed a $15 million Series A round led by London and Smedvig Capital, bringing Lunio’s total raised to around $17 million.

“Back in 2016, we were running a digital marketing agency in the U.K. and working closely with one of their top clients, Segev Hochberg,” Andrew told TechCrunch in an email interview. “During that time, they kept noticing the same problem. Worthless clicks from fake users were eating away a chunk of Segev’s marketing budget every month. And ad networks weren’t exactly rushing to tackle the problem, because there was no real incentive for them to do so. So Neil, Segev, and I founded Lunio to help other marketers catch and block clicks from bad sources, while automatically reinvesting the money saved back into top-performing ad campaigns.”

Lunio claims to use a combination of data analysis and cybersecurity techniques to catch and block fake clicks, with algorithms that run client-side — within a user’s browser — to ensure personally identifiable information isn’t sent over the web. (While the IP addresses of ad interactions are stored and provided to users, they’re not combined with any other information that could make them personally identifiable, Andrew claims.) The algorithms attempt to predict the likelihood of invalid click activity on a range of ad networks, including Google, YouTube, Facebook, Reddit, Instagram and even TikTok.


Image Credits: Lunio

While Lunio is far from the first click fraud prevention tech vendor — others include CHEQ and Human Security — Andrew asserts that its platform has key technical advantages. For example, Lunio’s algorithms leverage WebAssembly, the web standard designed to enable near-native code execution speed in the browser, which Andrew claims is up to seven times faster than traditional JavaScript — the programming language many vendors use to analyze ad traffic.

“There’s a huge opportunity cost of having distracted sales processes downstream due to fake lead form submissions which follow on from fake clicks. Sales reps can waste many hours chasing leads that don’t actually exist,” Andrew said. “It’s not just about getting a refund on spammy clicks — if you even manage to. It’s about stopping all the knock-on effects of having fake traffic hit your website.”

Andrew says the pandemic was a boon for Lunio because it led to increases in fake user activity as brands and their customers moved online. Meanwhile, the economic downturn has increased the pressure on companies to stretch their ad dollars, Andrew says — leading to another windfall for the startup.

Lunio has more than 1,000 customers covering over 10,000 individual advertising accounts, Andrew claims. He opted not to share revenue figures, saying only that Lunio plans to expand headcount from 43 employees to around 55 by the end of Q1 2023 to “accelerate” its go-to-market efforts in Europe and North America.

“We feel very insulated from the forward-looking challenges many companies will face. We have implemented strong operational and investment discipline based on validated business cases to drive our future direction,” Andrew continued. “We operate on a best-in-class burn multiple and expect this to continue in relative terms as we scale the business.”

Lunio raises $15M to combat click fraud with algorithms by Kyle Wiggers originally published on TechCrunch

Regie secures $10M to generate marketing copy using AI

Regie.ai, a startup using OpenAI’s GPT-3 text-generating system to create sales and marketing content for brands, today announced that it raised $10 million in Series A funding led by Scale Venture Partners with participation from Foundation Capital, South Park Commons, Day One Ventures and prominent angel investors. The fresh investment comes as VCs see a growing opportunity in AI-powered, copy-generating adtech companies, whose tech promises to save time while potentially increasing personalization.

Regie was founded in 2020 by Matt Millen and Srinath Sridhar. Previously a software engineer at Google and Meta, Sridhar is a data scientist by trade, having developed enterprise-scale AI systems that detect duplicate images and rank search results. Millen was formerly a VP at T-Mobile, leading the national sales teams (e.g., strategic accounts and public sector).

With Regie, Sridhar says he and Millen aimed to create a way for companies to communicate with their customers via channels like email, social media, text, podcasts, online advertising and more. Because companies have so many platforms and mediums at their disposal to speak with customers, he notes, it can be a challenge for content marketers to produce continuously compelling content to reach their customers.

“The way content is getting generated has fundamentally changed,” Sridhar told TechCrunch in an email interview. “Marketers and copywriters working in the enterprise … increasingly [need] to produce and manage content and content workflows at scale.”

Regie uses GPT-3 to power its service — the same GPT-3 that can generate poetry, prose and academic papers. But it’s a “flavor” of GPT-3 fine-tuned on a training data set of roughly 20,000 sales sequences (the series of steps to convert prospects into paying customers) and nearly 100 million sales emails. Also in the mix are custom language systems built by Regie to reflect brands and their messaging, designed to be integrated with existing sale platforms like Outreach, HubSpot, and Salesloft.


Image Credits: Regie

Lest the systems spew problematic language, Regie says that every system goes through “human curation” and vetting before being released. The startup also claims to train the systems on “inclusive” language and test them for biases, like bias against certain demographic groups.

Customers can use Regie to generate original, optimized-for-search-engines content or create custom sales sequences. The platform also offers blog- and social-media-post-authoring tools for personalizing messages, as well as a Chrome extension that analyzes the “quality” of emails that customers send — and optionally rewrites the text.

“Generative AI is completely disrupting the way content is created today. The biggest competitors of Regie would be the large content authoring and management platforms that will be completely redesigned AI first going forward,” Sridhar said confidently. “For example, Adobe’s suite of products including Acrobat, Illustrator, Photoshop, now Figma as well as Adobe Experience Cloud will start to get outdated as Regie continues to build on an intelligent content creation and management platform for the enterprise.”

More immediately, Regie competes with vendors like Jasper, Phrasee, Copysmith and Copy.ai — all of which tap AI to generate bespoke marketing copy. But Sridhar argues that Regie is a more vertical platform that caters to go-to-market teams in the enterprise while combining text, images and workflows into a single glass pane.

“Generative AI is such a paradigm shift that not only productivity and top-line of companies will go up as a result, but the bottom line will also go down simultaneously. There are very few products that can improve both sides of that financial equation,” Sridhar continued. “So if a company wants to reduce costs because they want to assimilate sales tools, or reduce outsourced writing while simultaneously increasing revenue, Regie can do that. If you are an outsourced marketing agency looking to retain more customers and efficiently generate content at scale, Regie can definitely do that for agencies as well.”

The company currently has more than 70 software-as-a-service customers on annual contracts, including AT&T, Sophos, Okta and Crunchbase. Sridhar didn’t reveal revenue but said that he expects the 25-person company to grow “meaningfully” this year.

“This is a revolutionary new field. And as always, adoption will require educating the users,” Sridhar said. “It is clear to us as practitioners that the world has changed. But it will take time for others to get their hands dirty and convince themselves that this is happening — and that it is a very positive development. So we have to be patient in educating the industry. We also have to show that content quality isn’t compromised and that it can perform better and be maintained more consistently with the strategic application of AI.”

To date, Regie has raised $14.8 million.

Regie secures $10M to generate marketing copy using AI by Kyle Wiggers originally published on TechCrunch

Google Search will soon begin translating local press coverage

At a Google Search-focused event this morning, Google announced that it will soon introduce ways to translate news coverage directly from Search. Starting in 2023, English users, for example, will be able to search and see translated links to news results from Spanish-language publishers in countries such as Mexico, in addition to links to articles written in their preferred language.

“Say you wanted to learn about how people in Mexico were impacted by the more than 7 magnitude earthquake earlier this month,” Google News product manager Itamar Snir and Google Search product manager Lauren Clark explained in a blog post. “With this feature, you’ll be able to search and see translated headlines for news results from publishers in Mexico, in addition to ones written in your preferred language. You’ll be able to read authoritative reporting from journalists in the country, giving you a unique perspective of what’s happening there.”

Google News translations

Image Credits: Google

Building off its earlier translation work, the feature will translate headlines and articles in French, German and Spanish into English to start on mobile and desktop.

Google has experimented with news translation before, three years ago adding the ability to display content in two languages together within the Google News app feed. But for the most part, the search giant has left it to users to translate content via tools like Chrome’s translate button and Google Translate. Presumably, should the Google Search news translation feature be well received, that’ll change for more languages in the future.

read more about Google Search On 2022 on TechCrunch

Google Search will soon begin translating local press coverage by Kyle Wiggers originally published on TechCrunch

OpenAI removes the waitlist for DALL-E 2, allowing anyone to sign up

Several months after launching DALL-E 2 as a part of a limited beta, OpenAI today removed the waitlist for the AI-powered image-generating system (which remains in beta), allowing anyone to sign up and begin using it. Pricing will remain the same, with first-time users getting a finite amount of credits that can be put toward generating or editing an image or creating a variation of existing images.

“More than 1.5 million users are now actively creating over 2 million images a day with DALL-E — from artists and creative directors to authors and architects — with about 100,000 users sharing their creations and feedback in our Discord community,” OpenAI wrote in a blog post. “Learning from real-world use has allowed us to improve our safety systems, making wider availability possible today.”

OpenAI has yet to make DALL-E 2 available through an API, though the company notes in the blog post that one is in testing. Brands such as Stitch Fix, Nestlé and Heinz have piloted DALL-E 2 for ad campaigns and other commercial use cases, but so far only in an ad hoc fashion.

As we’ve previously written about, OpenAI’s conservative release cycle appears intended to subvert the controversy growing around Stability AI’s Stable Diffusion, an image-generating system that’s deployable in an open source format without any restrictions. Stable Diffusion ships with optional safety mechanisms. But the system has been used by some to create objectionable content, like graphic violence and pornographic, nonconsensual celebrity deepfakes.

Stability AI — which already offers a Stable Diffusion API, albeit with restrictions on certain content categories — was the subject of a critical recent letter from U.S. House Representative Anna G. Eshoo (D-CA) to the National Security Advisor (NSA) and the Office of Science and Technology Policy (OSTP). In it, she urged the NSA and OSTP to address the release of “unsafe AI models” that “do not moderate content made on their platforms.”

Heinz DALL-E 2

Heinz bottles as “imagined” by DALL-E 2. Image Credits: Heinz

“I am an advocate for democratizing access to AI and believe we should not allow those who openly release unsafe models onto the internet to benefit from their carelessness,” Eshoo wrote. “Dual-use tools that can lead to real-world harms like the generation of child pornography, misinformation and disinformation should be governed appropriately.”

Indeed, as they march toward ubiquity, countless ethical and legal questions surround systems like DALL-E 2, Midjourney and Stable Diffusion. Earlier this month, Getty Images banned the upload and sale of illustrations generated using DALL-E 2, Stable Diffusion and other such tools, following similar decisions by sites including Newgrounds, PurplePort and FurAffinity. Getty Images CEO Craig Peters told The Verge that the ban was prompted by concerns about “unaddressed right issues,” as the training datasets for systems like DALL-E 2 contain copyrighted images scraped from the web.

The training data presents a privacy risk as well, as an Ars Technica report last week highlighted. Private medical records — possibly thousands — are among the many photos hidden within the dataset used to train Stable Diffusion, according to the piece. Removing these records is exceptionally difficult as LAION isn’t a collection of files itself but merely a set of URLs pointing to images on the web.

In response, technologists like Mat Dryhurst and Holly Herndon are spearheading efforts such as Source+, a standard aiming to allow people to disallow their work or likeness to be used for AI training purposes. But these standards are — and will likely remain — voluntary, limiting their potential impact.

DALL-E 2 Eric Silberstein

Experiments with DALL-E 2 for different product visualizations. Image Credits: Eric Silberstein

OpenAI has repeatedly claimed to have taken steps to mitigate issues around DALL-E 2, including rejecting image uploads containing realistic faces and attempts to create the likeness of public figures, like prominent political figures and celebrities. The company also says it trained DALL-E 2 on a dataset filtered to remove images that contained obvious violent, sexual or hateful content. And OpenAI says it employs a mix of automated and human monitoring systems to prevent the system from generating content that violates its terms of service.

“In the past months, we have made our filters more robust at rejecting attempts to generate sexual, violent and other content that violates our content policy, and building new detection and response techniques to stop misuse,” the company wrote in the blog post published today. “Responsibly scaling a system as powerful and complex as DALL-E — while learning about all the creative ways it can be used and misused — has required an iterative deployment approach.”

OpenAI removes the waitlist for DALL-E 2, allowing anyone to sign up by Kyle Wiggers originally published on TechCrunch

Kumo aims to bring predictive AI to the enterprise with $18M in fresh capital

Kumo, a startup offering an AI-powered platform to tackle predictive problems in business, today announced that it raised $18 million in a Series B round led by Sequoia, with participation from A Capital, SV Angel and several angel investors. Co-founder and CEO Vanja Josifovski says the new funding will be put toward Kumo’s hiring efforts and R&D across the startup’s platform and services, which include data prep, data analytics and model management.

Kumo’s platform works specifically with graph neural networks, a class of AI system for processing data that can be represented as a series of graphs. Graphs in this context refer to mathematical constructs made up of vertices (also called nodes) that are connected by edges (or lines). Graphs can be used to model relations and processes in social, IT and even biological systems. For example, the link structure of a website can be represented by a graph where the vertices stand in for webpages and the edges represent links from one page to another.

Graph neural networks have powerful predictive capabilities. At Pinterest and LinkedIn, they’re used to recommend posts, people and more to hundreds of millions of active users. But as Josifovski notes, they’re computationally expensive to run — making them cost-prohibitive for most companies.

“Many enterprises today attempting to experiment with graph neural networks have been unable to scale beyond training data sets that fit in a single accelerator (memory in a single GPU), dramatically limiting their ability to take advantage of these emerging algorithmic approaches,” he told TechCrunch in an email interview. “Through fundamental infrastructural and algorithmic advancements, we have been able to scale to datasets in the many terabytes, allowing graph neural networks to be applied to customers with larger and more complicated enterprise graphs, such as social networks and multi-sided marketplaces.”

Using Kumo, customers can connect data sources to create a graph neural network that can then be queried in structured query language (SQL). Under the hood, the platform automatically trains the neural network system, evaluating it for accuracy and readying it for deployment to production.

Josifovski says that Kumo can be used for applications like new customer acquisition, customer loyalty and retention, personalization and next best action, abuse detection and financial crime detection. Previously the CTO of Pinterest and Airbnb Homes, Josifovski worked with Kumo’s other co-founders, former Pinterest chief scientist Jure Leskovec and Hema Raghavan, to develop the graph technology through Stanford and Dortmund University research labs.

“Companies spend millions of dollars storing terabytes of data but are able to effectively leverage only a fraction of it to generate the predictions they need to power forward-looking business decisions. The reason for this is major data science capacity gaps as well as the massive time and effort required to get predictions successfully into production,” Josifovski said. “We enable companies to move to a paradigm in which predictive analytics goes from being a scarce resource used sparingly into one in which it is as easy as writing a SQL query, thus enabling predictions to basically become ubiquitous — far more broadly adapted in use cases across the enterprise in a much shorter timeframe.”

Kumo remains in the pilot stage, but Josifovski says that it has “more than a dozen” early adopters in the enterprise. To date, the startup has raised $37 million in capital.

Kumo aims to bring predictive AI to the enterprise with $18M in fresh capital by Kyle Wiggers originally published on TechCrunch

Document onboarding startup Flatfile nabs $50M from investors, including Workday

Data cleansing — prepping data for applications like predictive analytics — takes time. In fact, data scientists spend an estimated 60% of their time cleaning and organizing data, according to one recent survey. It’s not just time that’s lost. According to Experian, “dirty data” costs the average business 15% to 25% of their revenue and the U.S. economy $3 trillion annually.

On a mission to change things, Eric Crane and David Boskovic started Flatfile, a platform that automatically learns how imported data should be structured and cleaned. With customers like ClickUp, Square, AstraZeneca and Spotify, the startup is gearing up for its next growth phase, closing a $50 million Series B round that brings Flatfile’s total to $94.7 million.

Tiger Global led the Series B tranche with participation from GV (Google’s AI-focused fund) and Workday — the last of which no doubt saw the applicability of Flatfile’s data processing pipeline to its HR business. Scale Ventures and angel investors from Airtable, DocuSign, LinkedIn and Gainsight also contributed, Boskovic told TechCrunch in an email.

“Data exchange and onboarding the data of new customers in particular can take thousands of hours to complete as data is collected, cleaned and moved from one business to another,” Boskovic said. “Examples of this include clients sending bulk payments to a credit card company, or vendors sending supply chain updates to a food conglomerate. For large companies, data exchange can mean upwards of six months to prepare data causing delayed customer onboarding, cost overruns and lost clients … We envisioned a way to streamline the data exchange process to save them vast amounts of time and money.”

Crane and Boskovic created the tech behind Flatfile while at productivity startup Envoy, where they shared a mutual frustration with the many wasted hours spent manipulating and cleaning up the firm’s data. Through Flatfile, they sought specifically to address challenges in data onboarding, where the high variance across input files has historically made rule-based models ineffective.


Image Credits: Flatfile

Flatfile uses AI trained on over 25 billion “data decisions” to map and resolve schema with files such as spreadsheets and CSVs. When the algorithms encounter an anomaly or a data type they can’t process automatically, they prompt customers to make a decision and then add that scenario to a database for future reference.

Flatfile recently released a software development kit that will allow developers to build on top of Flatfile’s components to access import, match, merge and export functions. While the company continues to offer an out-of-the-box import workflow, the kit enables customers with more specific requirements to customize the experience, Boskovic said.

“It’s basically letting our customers get under the hood, allowing them to stitch together all the pieces required to move information between systems with maximum flexibility and at scale,” he added. “[The] platform enables companies to leverage their data sooner. It allows employees to focus on their core strengths and leave the dirty work to us. By eliminating the thousands of hours that companies consume ensuring that data is properly formatted for their system, Flatfile helps them get their products to market faster and at a substantial cost savings.”

Flatfile competes with incumbents like Textract, Amazon’s service that can automatically extract text and data from scanned documents, and Microsoft’s data onboarding tool Form Recognizer. Google offers its own data-extracting tools including Cloud Natural Language, which performs syntax, sentiment and entity analysis on existing files.

In any case, Boskovic says that the pandemic and economic downturn were huge growth opportunities for Flatfile — the pandemic because it led companies to migrate data to the cloud and the downturn because it put pressure on them to “prove their value faster.” Flatfile’s customer base stands at thousands of developers and 500 companies as well as several unnamed government organizations.

“Flatfile is in a strong position because it offers a comprehensive solution to a business critical challenge. While we had two years of runway left, we raised an opportunistic Series B to maximize on investor demand, [and now] we have four years of runway to continue improving our operations around customer feedback,” Boskovic said. “This investment will be used to expand and support Flatfile’s fastest growing segment: global enterprise companies. We have been rapidly growing over the last three quarters to reach about 75 employees, and we expect to continue this growth into the near future. Annual recurring revenue is over $5 million, and we project it will more than double over the next 12 months.”

Document onboarding startup Flatfile nabs $50M from investors, including Workday by Kyle Wiggers originally published on TechCrunch

Hugging Face and ServiceNow launch BigCode, a project to open source code-generating AI systems

Code-generating systems like DeepMind’s AlphaCode, Amazon’s CodeWhisperer and OpenAI’s Codex, which powers GitHub’s Copilot service, provide a tantalizing look at what’s possible with AI today within the realm of computer programming. But so far, only a handful of such AI systems have been made freely available to the public and open sourced — reflecting the commercial incentives of the companies building them.

In a bid to change that, AI startup Hugging Face and ServiceNow Research, ServiceNow’s R&D division, today launched BigCode, a new project that aims to develop “state-of-the-art” AI systems for code in an “open and responsible” way. The goal is to eventually release a data set large enough to train a code-generating system, which will then be used to create a prototype — a 15-billion-parameter model, larger in size than Codex (12 billion parameters) but smaller than AlphaCode (~41.4 billion parameters) — using ServiceNow’s in-house graphics card cluster. In machine learning, parameters are the parts of an AI system learned from historical training data and essentially define the skill of the system on a problem, such as generating code.

Inspired by Hugging Face’s BigScience effort to open source highly sophisticated text-generating systems, BigCode will be open to anyone who has a professional AI research background and can commit time to the project, say the organizers. The application form went live this afternoon.

“In general, we expect applicants to be affiliated with a research organization (either in academia or industry) and work on the technical/ethical/legal aspects of [large language models] for coding applications,” ServiceNow wrote in a blog post. “Once the [code-generating system] is trained, we’ll evaluate its capabilities … We’ll strive to make evaluation easier and broader so that we can learn more about the [system’s] capabilities.”

In collaboratively developing a code-generating system, which will be open sourced under a license that’ll allow developers to reuse it subject to certain terms and conditions, BigCode is seeking to address some of the controversies that have arisen around the practice of AI-powered code generation — particularly regarding fair use. The nonprofit Software Freedom Conservancy among others has criticized GitHub and OpenAI for using public source code, not all of which is under a permissive license, to train and monetize Codex. Codex is available through OpenAI’s paid API, while GitHub recently began charging for access to Copilot. For their parts, GitHub and OpenAI continue to assert that Codex and Copilot don’t run afoul of any license terms.

The BigCode organizers say they’ll take pains to ensure only files from repositories with permissive licenses go into the aforementioned training data set. Along they way, they say, they’ll work to establish “responsible” AI practices for training and sharing code-generating systems of all types, soliciting feedback from relevant stakeholders before making policy pronouncements.

ServiceNow and Hugging Face provided no timeline as to when the project might reach completion. But they expect it to explore several forms of code generation over the next few months, including systems that auto-complete and synthesize code from snippets of code and natural language descriptions and work across a wide range of domains, tasks and programming languages.

Assuming the ethical, technical and legal issues are someday ironed out, AI-powered coding tools could cut development costs substantially while allowing coders to focus on more creative tasks. According to a study from the University of Cambridge, at least half of developers’ efforts are spent debugging and not actively programming, which costs the software industry an estimated $312 billion per year.

Hugging Face and ServiceNow launch BigCode, a project to open source code-generating AI systems by Kyle Wiggers originally published on TechCrunch

Perceptron: Multilingual, laughing, Pitfall-playing and streetwise AI

Research in the field of machine learning and AI, now a key technology in practically every industry and company, is far too voluminous for anyone to read it all. This column, Perceptron, aims to collect some of the most relevant recent discoveries and papers — particularly in, but not limited to, artificial intelligence — and explain why they matter.

Over the past few weeks, researchers at Google have demoed an AI system, PaLI, that can perform many tasks in over 100 languages. Elsewhere, a Berlin-based group launched a project called Source+ that’s designed as a way of allowing artists, including visual artists, musicians and writers, to opt into — and out of — allowing their work being used as training data for AI.

AI systems like OpenAI’s GPT-3 can generate fairly sensical text, or summarize existing text from the web, ebooks and other sources of information. But they’re historically been limited to a single language, limiting both their usefulness and reach.

Fortunately, in recent months, research into multilingual systems has accelerated — driven partly by community efforts like Hugging Face’s Bloom. In an attempt to leverage these advances in multilinguality, a Google team created PaLI, which was trained on both images and text to perform tasks like image captioning, object detection and optical character recognition.

Google PaLI

Image Credits: Google

Google claims that PaLI can understand 109 languages and the relationships between words in those languages and images, enabling it to — for example — caption a picture of a postcard in French. While the work remains firmly in the research phases, the creators say that it illustrates the important interplay between language and images — and could establish a foundation for a commercial product down the line.

Speech is another aspect of language that AI is constantly improving in. Play.ht recently showed off a new text-to-speech model that puts a remarkable amount of emotion and range into its results. The clips it posted last week sound fantastic, though they are of course cherry-picked.

We generated a clip of our own using the intro to this article, and the results are still solid:

Exactly what this type of voice generation will be most useful for is still unclear. We’re not quite at the stage where they do whole books — or rather, they can, but it may not be anyone’s first choice yet. But as the quality rises, the applications multiply.

Mat Dryhurst and Holly Herndon — an academic and musician, respectively — have partnered with the organization Spawning to launch Source+, a standard they hope will bring attention to the issue of photo-generating AI systems created using artwork from artists who weren’t informed or asked permission. Source+, which doesn’t cost anything, aims to allow artists to disallow their work to be used for AI training purposes if they choose.

Image-generating systems like Stable Diffusion and DALL-E 2 were trained on billions of images scraped from the web to “learn” how to translate text prompts into art. Some of these images came from public art communities like ArtStation and DeviantArt — not necessarily with artists’ knowledge — and imbued the systems with the ability to mimic particular creators, including artists like Greg Rutowski.

Stability AI Stable Diffusion

Samples from Stable Diffusion.

Because of the systems’ knack for imitating art styles, some creators fear that they could threaten livelihoods. Source+ — while voluntary — could be a step toward giving artists greater say in how their art’s used, Dryhurst and Herndon say — assuming it’s adopted at scale (a big if).

Over at DeepMind, a research team is attempting to solve another longstanding problematic aspect of AI: its tendency to spew toxic and misleading information. Focusing on text, the team developed a chatbot called Sparrow that can answer common questions by searching the web using Google. Other cutting-edge systems like Google’s LaMDA can do the same, but DeepMind claims that Sparrow provides plausible, non-toxic answers to questions more often than its counterparts.

The trick was aligning the system with people’s expectations of it. DeepMind recruited people to use Sparrow and then had them provide feedback to train a model of how useful the answers were, showing participants multiple answers to the same question and asking them which answer they liked the most. The researchers also defined rules for Sparrow such as “don’t make threatening statements” and “don’t make hateful or insulting comments,” which they had participants impose on the system by trying to trick it into breaking the rules.

Example of DeepMind’s sparrow having a conversation.

DeepMind acknowledges that Sparrow has room for improvement. But in a study, the team found the chatbot provided a “plausible” answer supported with evidence 78% of the time when asked a factual question and only broke the aforementioned rules 8% of the time. That’s better than DeepMind’s original dialogue system, the researchers note, which broke the rules roughly three times more often when tricked into doing so.

A separate team at DeepMind tackled a very different domain recently: video games that historically have been tough for AI to master quickly. Their system, cheekily called MEME, reportedly achieved “human-level” performance on 57 different Atari games 200 times faster than the previous best system.

According to DeepMind’s paper detailing MEME, the system can learn to play games by observing roughly 390 million frames — “frames” referring to the still images that refresh very quickly to give the impression of motion. That might sound like a lot, but the previous state-of-the-art technique required 80 billion frames across the same number of Atari games.

DeepMind MEME

Image Credits: DeepMind

Deftly playing Atari might not sound like a desirable skill. And indeed, some critics argue games are a flawed AI benchmark because of their abstractness and relative simplicity. But research labs like DeepMind believe the approaches could be applied to other, more useful areas in the future, like robots that more efficiently learn to perform tasks by watching videos or self-improving, self-driving cars.

Nvidia had a field day on the 20th announcing dozens of products and services, among them several interesting AI efforts. Self-driving cars are one of the company’s foci, both powering the AI and training it. For the latter, simulators are crucial and it is likewise important that the virtual roads resemble real ones. They describe a new, improved content flow that accelerates bringing data collected by cameras and sensors on real cars into the digital realm.

A simulation environment built on real-world data.

Things like real-world vehicles and irregularities in the road or tree cover can be accurately reproduced, so the self-driving AI doesn’t learn in a sanitized version of the street. And it makes it possible to create larger and more variable simulation settings in general, which aids robustness. (Another image of it is up top.)

Nvidia also introduced its IGX system for autonomous platforms in industrial situations — human-machine collaboration like you might find on a factory floor. There’s no shortage of these, of course, but as the complexity of tasks and operating environments increases, the old methods don’t cut it any more and companies looking to improve their automation are looking at future-proofing.

Example of computer vision classifying objects and people on a factory floor.

“Proactive” and “predictive” safety are what IGX is intended to help with, which is to say catching safety issues before they cause outages or injuries. A bot may have its own emergency stop mechanism, but if a camera monitoring the area could tell it to divert before a forklift gets in its way, everything goes a little more smoothly. Exactly what company or software accomplishes this (and on what hardware, and how it all gets paid for) is still a work in progress, with the likes of Nvidia and startups like Veo Robotics feeling their way through.

Another interesting step forward was taken in Nvidia’s home turf of gaming. The company’s latest and greatest GPUs are built not just to push triangles and shaders, but to quickly accomplish AI-powered tasks like its own DLSS tech for uprezzing and adding frames.

The issue they’re trying to solve is that gaming engines are so demanding that generating more than 120 frames per second (to keep up with the latest monitors) while maintaining visual fidelity is a Herculean task even powerful GPUs can barely do. But DLSS is sort of like an intelligent frame blender that can increase the resolution of the source frame without aliasing or artifacts, so the game doesn’t have to push quite so many pixels.

In DLSS 3, Nvidia claims it can generate entire additional frames at a 1:1 ratio, so you could be rendering 60 frames naturally and the other 60 via AI. I can think of several reasons that might make things weird in a high performance gaming environment, but Nvidia is probably well aware of those. At any rate you’ll need to pay about a grand for the privilege of using the new system, since it will only run on RTX 40 series cards. But if graphical fidelity is your top priority, have at it.

Illustration of drones building in a remote area.

Last thing today is a drone-based 3D printing technique from Imperial College London that could be used for autonomous building processes sometime in the deep future. For now it’s definitely not practical for creating anything bigger than a trash can, but it’s still early days. Eventually they hope to make it more like the above, and it does look cool, but watch the video below to get your expectations straight.

Perceptron: Multilingual, laughing, Pitfall-playing and streetwise AI by Kyle Wiggers originally published on TechCrunch

Tech is at the heart of the biggest chess drama in years

The chess world is currently consumed by a drama as lurid and compelling, in its way, as the Don’t Worry Darling fracas. Involving implications of cheating at the highest levels of play, the feud between the world champion and an upstart challenger has prompted speculation on the existential threat to chess posed by an AI engine tiny enough to be concealed somewhere on — or in — the body.

The idea — unsupported by any evidence, it must be emphasized — that a player could surreptitiously consult an unbeatable chess engine even when playing over the board has been batted around for years. But recent events have made people think seriously about the possibility and what it might mean for the future of the game.

The saga began two weeks ago, when current chess world champion and one of the strongest players in history, Magnus Carlsen (pictured above), began a match at the Sinquefield Cup with Hans Niemann, a 19-year-old grandmaster who has ascended from respectable to downright dangerous over a remarkably short period of time.

Carlsen was playing with the white pieces, and therefore going first — an advantage he is particularly adept at using, having not lost a game in years with white and seldom even taking a draw. Yet soon he had not just forfeit the game (which you can watch here), he had withdrawn from the tournament, cryptically tweeting what seemed to many took to be a veiled accusation of cheating by Niemann. He has not elaborated on these actions despite officials, fans, colleagues and even the likes of former world champion Garry Kasparov, asking him to speak out.

Niemann, for his part, has naturally and emphatically denied any cheating, and said that an apparently miraculous preparation for the unusual line of attack Carlsen took was one he happened to include after seeing it in a game from years earlier. The tournament organizer has stated that there is no indication of any suspicious behavior or wrongdoing. Others have examined the record and found no indication of cheating.

The event and its fallout (only given in outline here) have sharply divided the chess world, as even the conservative approach of “let’s wait and see” tacitly sustains the idea of Niemann cheating, so there is precious little neutral ground to occupy. FIDE, the official international chess organization, is expected to issue a statement soon that may shed light on things, but it won’t change what’s already happened.

A checkered history

Promotional image from a recent matchup between Carlsen (left) and Niemann.

To be clear here, there is no question that Niemann is an extremely high-level player — he has played hundreds of games against extremely strong players in situations where cheating is all but impossible and won decisively.

It was noted by other GMs that Carlsen had played poorly (for him), and Niemann had simply gotten lucky with his prep, played well and perhaps rattled the champion, leading to an advantageous position. But Carlsen is not easily rattled, nor is he one to storm off after making a blunder — still less to cast unfounded aspersions on an opponent. They have faced off before — one fan even caught them playing a friendly barefoot match on the beach in Miami just weeks earlier.

But Niemann, like anyone, has a past. It came out that Carlsen was likely aware that Niemann has been caught cheating online before — at least twice on Chess.com, when he was 12 and 16 respectively, the latter time seemingly leading to a six-month suspension from prize games. He admitted this in an interview, calling it the foolishness of a young, ambitious player — though he is still young and ambitious — and that it involved asking someone else in the room to furnish him with moves from a chess engine.

I was just a child. I have never ever in my life cheated in an over-the-board game. I wanted to gain some rating so I could play stronger players, so I cheated in random games in chess.com. I was confronted, and I confessed. It was the single biggest mistake of my life and I am completely ashamed.

While there are measures against this type of thing online, they are hardly foolproof. Chess.com has implied that there was more to Niemann’s behavior than this, but the details are not public.

Of course it is behavior unbecoming of a grandmaster, but whatever you make of his character, the fact is that cheating online is relatively easy — cheating over the board is practically Mission: Impossible. Cheating versus the world champion — at a disadvantage — at a major tournament? The very idea is ludicrous.

Or is it?

The chess community at large, a diverse group of players and commentators of all ages and skill levels, could not help but think about how, if one were insane enough to try to cheat in an over the board game with Magnus holding the white pieces, how would you do it? Purely theoretically, for argument’s sake, devil’s advocate and all that?

It turns out to be not quite so ludicrous as one might think — and there is just enough (admittedly highly circumstantial) evidence to admit a shadow of doubt.

How tech crept in

Illustration of chess pieces floating in space with video game bombs.

DeepMind’s MuZero is an example of a generalized game-playing neural network strong enough to dominate human players. Image Credits: DeepMind

The funny thing about cheating in high-level chess is the idea that the player would need help in the first place.

In a match between grandmasters, who exactly is one of them going to ask for advice? They can’t find a confederate to wink at them from the galleries: Nearly everyone in the world is worse at the game. Chess is more popular than ever, but nevertheless the number of people playing at Carlsen and Niemann’s level is in the dozens. Collusion is unthinkable.

A chess engine, however, plays at an even higher level. You may hear the phrase and think of Deep Blue and Kasparov, the man versus the supercomputer, but nowadays engines infinitely superior to Deep Blue are available on any smartphone. Indeed, one could conceivably run one on a tiny computer like a Raspberry Pi Zero. Something you could slip into a pocket, or a shoe, or perhaps somewhere no one would think of looking.

Players undergo strict security measures, of course, and you can see Niemann himself being examined here. The possibility is taken seriously, but advances in technology always move faster than countermeasures.

The difficulty presented by such a theoretical device is twofold.

First, how would it even know the state of the game? After all, they’re playing with real pieces over a real board. We can dispense with the idea of wiggling toes or tapping feet to select moves — it becomes impractical (and easily noticeable) very quickly. But as it turns out, most OTB games at this level are streamed online and notated in real time — very shortly after a player places their piece, a virtual board is updated and the move is registered online for others to discuss, play along with, and so on. It would be trivial to pull this information from online, passed to the device wirelessly.

It happens that, according to at least one analysis, Niemann has performed better on OTB games streaming live in this way, and more poorly on ones that aren’t (some dispute the analysis, or offer other reasons for this). And in the case of the game against Carlsen, shortly after his forfeit the stream was placed on a 15-minute delay, which eliminates the possibility of cheating in this way. Odd, but far from conclusive — hardly even suggestive to anyone not already suspicious.

The second difficulty is how the device would communicate its suggestions to the player. One can hardly see a screen with the engine proposing various lines, but you don’t need to. Chess is efficiently notated: Qh5, for instance, means Queen to white’s far right column for white, fifth row up. People pointed out that a handful of short signals, in Morse code or the like, could provide complete information.

Let us admit that seems a little far-fetched — imagine a grandmaster attempting to look like they’re focusing on the game while the engine in their shoe stutters out a handful of promising defenses. In fact it’s been tried and detected. But the truth is it’s much simpler than that: As champions of the game have said for decades, anyone good enough doesn’t need to be told what play to make — only that there is a play to make.

“All I would need in order to be almost invincible”

Reviewing their matches, even the greatest players spot moments where, had they seen a given line of attack or defense, they could have crushed their opponent or snatched a draw out of the jaws of certain defeat. It’s the chess version of “l’esprit d’escalier,” when you think of the perfect comeback to some jibe hours later as you climb the stairs to bed.

If a player at the grandmaster level could rely on being told, even once in a game, that this move was potentially crucial, they would be practically unbeatable. There’s no need for Morse code — the simplest of signals would suffice to inform the player that there is a play to be made, trusting to their skill to find it.

Carlsen expressed this himself in a (translated from Norwegian) 2021 interview:

The people who get caught are those who cheat in a really obvious and stupid manner. The problem was that he [i.e. a player caught in 2016] was not good enough to see what would’ve made sense.

Had I started cheating in a clever manner, I am convinced no one would notice. I would’ve just needed to cheat one or two times during the match, and I would not even need to be given moves, just the answer on which move was way better. Or, here there is a possibility of winning, and here you need to be more careful. That is all I would need in order to be almost invincible, which does frighten me.

At the end of the day, the game doesn’t work if you do not trust your opponents. I’m not going to sit here and spread rumors, but it would not surprise me at all if we’ve had a lot of cheaters, even in big tournaments, that have won and not been caught.

And here we find another little quirk of Niemann’s: He is occasionally not great at explaining his chess. Post-game analysis is an enormous part of chess commentary, and players frequently discuss positions, moves and alternatives. In discussion with others who play at his level, Niemann occasionally appears (to others in a position to know) unable to express the reasoning behind a move, what led to it or where it would lead.

This is not that strange in and of itself. Chess is both analytical and intuitive, but flashes of insight may not be equally well remembered by all brains, especially neurodivergent ones common to the game. Not everyone has the expected clinical, comprehensive viewpoint associated with the mindset — as chess has grown, it has embraced new approaches and personalities. Niemann is one such personality, outspoken and opinionated, streaming and tweeting and generally taking part in the discourse like any talented 19-year-old might with their favored community. His way of communicating his chess doesn’t have to match what is expected of him.

But in the context of the recent drama, this occasional incapacity to explain his own thought process has been counted against him by his detractors.

Chess will survive tech (again)

There are two other obvious alternative explanations for all of this: First, that Niemann simply beat Carlsen fair and square and this is all a big misunderstanding (though one that would be devastating to Carlsen’s reputation for several reasons). The second — for which, again, there is no evidence — is that someone leaked Carlsen’s strategy to his opponent, a much more prosaic form of cheating that requires no technology whatsoever.

Should either of these be the case, the Hans-Magnus kerfuffle has still let the genie (back) out of the bottle. High-tech cheating has been an issue for years, basically since chess engines passed a human level of play. Commentators have considered it even in some high-level games but credible accusations remain few and far between. Security measures like metal detectors, banning of all devices at venues, delays on game broadcasts, and so on have been put in place to stop the obvious methods. Yet the possibility remains.

One only has to spend a few minutes thinking of methods to do it with today’s technology to conceive of something technically doable, and subtle enough that no one would suspect anything strange was happening. As Carlsen said, a clever cheater would be invincible if they were good enough to compete in the first place. It would not show up in statistical analysis or trip the intuition of strong players, both of which are finely tuned to detecting computer-type chess. (AI’s style is inimitable, it seems, in chess and other games it has come to dominate.)

As I was writing this piece, Carlsen and Niemann faced off again in a live-streamed game; Carlsen forfeited after two moves, stunning the chess world and quickly prompting criticism from his peers. It’s one thing to harbor suspicions, they said, but to participate and deliberately forfeit a game like that is dishonorable and unnecessary and puts his status as world champion in jeopardy. But others took it as the action of someone who can’t say what he knows and would rather lose ignominiously than play in bad conscience. (There is speculation he has presented his case to FIDE and awaits their decision, and is prohibited from discussing it publicly. Indeed neither of them has posted to Twitter in weeks.)

It is potentially a crisis of confidence in the chess world — the specter of cheating, always present but seldom mentioned, is suddenly in every headline. Such a reckoning may lead to major changes in the chess world on the order of how chess engines did two decades ago. Chess, of course, will remain — but just as players had to learn that they would never be as good as an engine, they may have to accept that undetectable cheating at the GM level is at best possible and at worst systemic.

How will that change the game and community? Many thought that, following Kasparov’s defeat at the hands of Deep Blue, humanity would lose its taste for a game it couldn’t win. In fact the opposite happened and the chess scene has become even more vibrant, the level of play higher than ever. Could the same thing happen with the idea that an AI may be concealed in one’s opponent’s shoe, their tooth, their watch? It’s only outlandish until you find out someone has been getting away with it for years.

The drama is still unfolding and it may in fact be less far-reaching than this. But the community can’t forget and it must reckon with the possibilities it contemplated, if only in theory. Chess will survive and thrive, but it will never be the same again

Tech is at the heart of the biggest chess drama in years by Devin Coldewey originally published on TechCrunch