The Dutch data protection agency has asked Microsoft’s lead privacy regulator in Europe to investigate ongoing concerns it has attached to how Windows 10 gathers user data.
Back in 2017 the privacy watchdog found Microsoft’s platform to be in breach of local privacy laws on account of how it collects telemetry metadata.
After some back and forth with the regulator, Microsoft made changes to how the software operates in April last year — and it was in the course of testing those changes that the Dutch agency found fresh reasons for concern, discovering what it calls in a press release “new, potentially unlawful, instances of personal data processing”.
Since the agency’s investigation of Windows 10 started a new privacy framework is being enforced in Europe — the General Data Protection Regulation (GDPR) — which means Microsoft’s lead EU privacy regulator is the Irish Data Protection Commission (DPC), where its regional HQ is based. This is why the Dutch agency has referred its latest concerns to Ireland.
It will now be up to the Irish DPC to investigate Windows 10, adding to its already hefty stack of open files on multiple tech giants’ cross-border data processing activities since the GDPR came into force last May.
The regulation steps up the penalties that can be imposed for violations (to up to 4% of a company’s annual global turnover).
A spokeswoman for the Irish DPC confirmed to TechCrunch that it received the Dutch agency’s concerns last month. “Since then the DPC has been liaising with the Dutch DPA to further this matter,” she added. “The DPC has had preliminary engagement with Microsoft and, with the assistance of the Dutch authority, we will shortly be engaging further with Microsoft to seek substantive responses on the concerns raised.”
A Microsoft spokesperson also told us:
The Dutch data protection authority has in the past brought data protection concerns to our attention, which related to the consumer versions of Windows 10, Windows 10 Home and Pro. We will work with the Irish Data Protection Commission to learn about any further questions or concerns it may have, and to address any further questions and concerns as quickly as possible.
Microsoft is committed to protecting our customers’ privacy and putting them in control of their information. Over recent years, in close coordination with the Dutch data protection authority, we have introduced a number of new privacy features to provide clear privacy choices and easy-to-use tools for our individual and small business users of Windows 10. We welcome the opportunity to improve even more the tools and choices we offer to these end users.
The Dutch DPA advises users of Windows 10 to pay close attention to privacy settings when installing and using the software.
“Microsoft is permitted to process personal data if consent has been given in the correct way,” it writes. “We’ve found that Microsoft collect diagnostic and non-diagnostic data. We’d like to know if it is necessary to collect the non-diagnostic data and if users are well informed about this.
“Does Microsoft collect more data than they need to (think about dataminimalization as a base principle of the GDPR). Those questions can only be answered after further examination.”
During the onboarding process for Windows 10, Microsoft makes multiple requests to process user data for various reasons, including ad purposes.
It also deploys the female voice of Cortana, its digital assistant technology, to provide a running commentary on settings screens — which can include some suggestive prompts to agree to its T&Cs. “If you don’t agree, y’know, no Windows!” the human-sounding robot says at one point. It’s not clear whether the Dutch agency’s concerns extend to Microsoft’s use of Cortana to nudge users during the Windows 10 consent flow.
Microsoft is the latest tech giant to amend its privacy policy after media reports revealed it uses human contractors to review audio recordings of Skype and Cortana users.
A section in the policy on how the company uses personal data now reads (emphasis ours):
Our processing of personal data for these purposes includes both automated and manual (human) methods of processing. Our automated methods often are related to and supported by our manual methods. For example, our automated methods include artificial intelligence (AI), which we think of as a set of technologies that enable computers to perceive, learn, reason, and assist in decision-making to solve problems in ways that are similar to what people do. To build, train, and improve the accuracy of our automated methods of processing (including AI), we manually review some of the predictions and inferences produced by the automated methods against the underlying data from which the predictions and inferences were made. For example, we manually review short snippets of a small sampling of voice data we have taken steps to de-identify to improve our speech services, such as recognition and translation.
The tweaks to the privacy policy of Microsoft’s Skype VoIP software and its Cortana voice AI were spotted by Motherboard — which was also first to report that contractors working for Microsoft are listening to personal conversations of Skype users conducted through the app’s translation service, and to audio snippets captured by the Cortana voice assistant.
Asked about the privacy policy changes, Microsoft told Motherboard: “We realized, based on questions raised recently, that we could do a better job specifying that humans sometimes review this content.”
Multiple tech giants’ use of human workers to review users’ audio across a number of products involving AI has grabbed headlines in recent weeks after journalists exposed a practice that had not been clearly conveyed to users in terms and conditions — despite European privacy law requiring clarity about how people’s data is used.
Apple, Amazon, Facebook, Google and Microsoft have all been called out for failing to make it clear that a portion of audio recordings will be accessed by human contractors.
Such workers are typically employed to improve the performance of AI systems by verifying translations and speech in different accents. But, again, this human review component within AI systems has generally been buried rather than transparently disclosed.
Earlier this month a German privacy watchdog told Google it intended to use EU privacy law to order it to halt human reviews of audio captured by its Google Assistant AI in Europe — after press had obtained leaked audio snippets and being able to re-identify some of the people in the recordings.
On learning of the regulator’s planned intervention Google suspended reviews.
Facebook also said it was pausing human reviews of a speech-to-text AI feature offered in its Messenger app — again after concerns had been raised by journalists.
So far Apple, Google and Facebook have suspended or partially suspended human reviews in response to media disclosures and/or regulatory attention.
In response to the rising privacy scrutiny of what tech giants nonetheless claim is a widespread industry practice, Amazon also recently amended the Alexa privacy policy to disclose that it employs humans to review some audio. It also quietly added an option for uses to opt-out of the possibility of someone listening to their Alexa recordings. Amazon’s lead EU privacy regulator is also now seeking answers.
Microsoft told Motherboard it is not suspending human reviews at this stage.
Users of Microsoft’s voice assistant can delete recordings — but such deletions require action from the user and would be required on a rolling basis as long as the product continues being use. So it’s not the same as having a full and blanket opt out.
We’ve asked Microsoft whether it intends to offer Skype or Cortana users an opt out of their recordings being reviewed by humans.
The company told Motherboard it will “continue to examine further steps we might be able to take”.
Their dangers are becoming more known and understood. The media is rife with articles detailing the speed at which the technology has grown in sophistication and become more accessible, as well as the risks involved.
Good.
The negative implications of deepfakes are troubling, and the better we understand them, the better we’ll be able to prevent their worst consequences. For better or worse, the technology is here to stay. But there is a “better” here—deepfakes have much in the way of lighthearted upside.
Though the debate around deepfakes has grown in stature and complexity, we still struggle to agree on a definition of deepfakes. I think of it as any mimicry, manipulation, or synthesis of video or audio that is enabled by machine learning. Face-swapping, body puppetry, copying someone’s voice, and creating entirely new voices or images all fall into this category. Your Photoshop efforts, valiant though they are, don’t.
Image synthesis and manipulation can be a powerful tool for creators
Visual storytelling is an expensive business. Hollywood studios spend billions on creating spectacle that wows their audience or transports them to another world. The tools they use to do so—the tools these big players use to close the gap between what they can imagine and what they can create—remain prohibitively expensive for most creators, though less so than a decade ago. Deepfake tech incorporates the ability to synthesize imagery, potentially giving smaller-scale creators a similar capacity for bringing imaginative creativity to life.
Synthesia is a company with a commercial product that uses deepfake tech to do automated and convincing dubbing through automated facial re-animation. They shot to prominence with a video that featured David Beckham talking about Malaria in nine languages, but their product could also be used to expand the reach of creators around the world. If you’re a talented artist who isn’t working in one of the world’s dominant languages, it’s potentially career-changing to have access to a product like this, which could make your work viable in additional languages and countries.
Adobe VoCo is software — albeit still at a research and prototyping stage — that makes it easier for creators to produce speech from text and edit it the way they would edit images in Photoshop. So if you want your movie short to be narrated by Morgan Freeman, you might be able to make that happen.
Tinghui Zhou, the founder and CEO of Humen, a company that creates deepfakes for dancing, sums up the industry’s goals: “The future we are imagining is one where everyone can create Hollywood-level content.” (Disclosure: I am an investor in Humen).
In the same way YouTube and Instagram shrunk the distribution and creation advantage that entertainment companies and famous photographers enjoyed over talented amateurs and enthusiasts, this bundle of technologies might diminish the production advantage currently possessed by big budgets and visual effects houses.
Mimicry and manipulation of real life have always been part of art.
The applications mentioned above are all to do with closing the gap between creators with different resources, but deepfake tech could also enable entirely new forms of content that rest on the ability to mimic and manipulate material. Every medium of entertainment has incorporated the stretching, reflection, contortion, and appropriation of real source material for the purposes of entertainment.
We can already see the evidence of these new applications in the still-nascent use of deepfake tech today. While face swapping for porn lies at the malicious end of the spectrum, more benignly the technology’s introduction also sparked a wave of face swapping Nicolas Cage into different movies.
It might seem banal, but it was a form of content creation that, while previously technically possible, was practically infeasible before deepfakes. It’s not hard to imagine that the next deepfakes content craze will be driven by automated lip-syncing, dance mimicry, orcelebrity voice impressions.
Respeecher and Replica.AI are just two companies making voice mimicry accessible to non-techies. Check out my demo with Replica’s tech in San Francisco a few weeks ago (recognize the voice?). It’s a small slice of the future of entertainment and content. If you believe that culture in the digital era is the culture of remixing, then deepfake tech has an important part to play in the creation of that culture.
Deepfakes bring us closer to believable virtual humans
The ability to mimic faces, voices, and emotional expressions is one of the most important steps toward building a believable virtual human that we can actually interact with. We’re already taking tentative steps down the path to virtual humans. Personal assistants like Alexa, Siri, and Cortana have been around for several years, reached a tipping point of consumer use, and are quickly improving. Having said that, in 2019 they still feel more like a new user interface you have to pass precise instructions to rather than a virtual being you can interact with. Think a command line operated by speech.
Virtual humans are entering the mainstream in a different way: Through the recent wave of digital influencers. I previously wrote about this trend in the context of animation history, but digital influencers are also meaningful in the context of believable virtual humans. Digital influencers operate on the same planes of interaction — think your Instagrams and Pinterests — that most people do.
As such, you and I can comment on a Lil Miquela post or message Astro. This is interaction with a being that isn’t real. The digital influencer isn’t really responding to you in their own words — their content is created by storytellers, much as Pixar films have writers. But these digital influencers are laying the social groundwork for interaction with true virtual beings.
Deepfakes have the potential to plug the technological holes in smart assistants and digital influencers. Pushing Alexa or Lil Miquela to the level of virtual humans like Samantha from Her or Joi from Bladerunner 2049 requires the capacity to encompass and express human body language, speech, and emotion. If we counted the number of unique combinations of pose, vocal nuance, and facial expressions you’ve made in your lifetime, it would likely number in the billions. For virtual humans to be believable, their actions can’t be preprogrammed in a traditional hard-coded sense, but must instead be extremely flexible.
Deepfake tech typically takes tons of examples of human behavior as inputs and then produces outputs that approximate or elaborate on that behavior. It could grant smart assistants the capacity to understand and originate conversation with much more sophistication. Similarly, digital influencers could develop the ability to visually react in a believable way in real time, thanks to deepfake tech. Bringing Mickey Mouse to life beyond a Disney cartoon or guy in a suit at Disneyland is where we’re headed. 3D hologram projections of animated characters (and real people) that are able to speak in a realistic sounding voice, moving like their real world counterpart would.
Creativity starts with copying. Elaboration follows duplication. It is no different with deepfakes, which will democratize access to creativity tools in entertainment, enable entirely new forms of content, and bring us closer to believable digital humans. That is why I think there is as much reason to be excited about the technology’s virtues as there is to be concerned about its vices.
The Harman Kardon Invoke was fine. But let’s be real — the first Cortana smart speaker was dead on arrival. Microsoft’s smart assistant has its strong suits, but thus far statement of purpose hasn’t been among them. CEO Satya Nadella appears to have acknowledge as much this week during a media event at the company’s Redmond Campus.
“Defeat” might be a strong word at this stage, but the executive is publicly acknowledging that the company needs to go back to the drawing board. In its current configuration, the best Microsoft can seemingly hope for with Cortana is a slow ramp up after a greatly delayed start. For all of the company’s recent successes, the gulf between its offering and Alexa, Assistant (and to a lesser degree) Siri must seem utterly insurmountable.
The new vision for Cortana is an AI offering that works in tandem with products that have previously been considered its chief competitors. That’s in line with recent moves. Over the summer, Microsoft and Amazon unveiled integration between the two assistants. Nadella used this week’s event to both reaffirm plans to work with Alexa and Google Assistant and note that past categories probably don’t make sense, going forward.
“We are very mindful of the categories we enter where we can do something unique,” he told the crowd. “A good one is speakers. To me the challenge is, exactly what would we be able to do in that category that is going to be unique?”
It’s a fair question. And the answer, thus far, is nothing. Like Samsung’s Bixby offerings, the primary distinguisher has been the devices its chosen to roll out on — appliances for Bixby and PCs for Microsoft. And while moves by Apple, Amazon and Google have all been acknowledgements that desktops and laptops may play an important role in the growth of smart assistants moving forward, but they were hardly a major driver early on.
I suspect this will also means the company will invest less in pushing Cortana as a consumer-facing product for the time being, instead focusing on the ways it can help other more popular assistants play nicely with the Microsoft ecosystem.
