Report: Apple explored ads in Apple Maps, as it weighs broader ads push

A new report from Bloomberg indicates Apple may be expanding its advertising business beyond the App Store and other first-party apps like News and Stocks. This plan could see Apple introducing ads into other pre-installed, first-party apps like Apple Maps, where ads have already been explored internally, the report claimed. And later, Apple may roll out ads to other Apple apps, like Books and Podcasts.

The additions could help increase the annual revenue for Apple’s ads business, currently $4 billion, to reach the double digits, the report noted, making Apple a more sizable player in the digital ads industry.

Today, Apple serves a variety of ads across its App Store, including display ads on the Store’s Search tab in addition to the Search Ads that appear at the top of the search results when users type in specific keywords to locate apps. Recently, Apple was said to be expanding its selection of App Store advertising slots to also include ads on the main Today tab and at the bottom of third-party apps’ listing in the “You Might Also Like” section of app recommendations.

Elsewhere, Apple runs display ads within its first-party News and Stocks apps and it broadcast ads within its Apple TV+ live streams of MLB games and pre-game shows. (It shares a cut of its News ad revenue with publishers, we should note.)

According to Mark Gurman’s Bloomberg Newsletter, Apple has already internally tested Search Ads within Maps. These work similarly to the App Store Search Ads, he says, as they would allow an advertiser to bid on keywords in order to have their business returned when a user searched for a particular term.

Meanwhile, the report suggests publishers could pay to have their work appear higher in search results in Apple’s Books and Podcasts apps or these apps could include display ads. It did not indicate if either of these apps had seen ad tests, however.

Apple TV+ could also prove to be another sizable ad platform, the report also speculated, arriving at a time when major streamers like Disney+ and Netflix have introduced more affordable ad-supported plans. Or Apple could offer introduce some sort of free streaming hub supported by ads. This could compete with other free movie and TV streaming offerings like those provided by The Roku Channel, Tubi, or Amazon’s Freevee, perhaps.

Further, Bloomberg noted that Apple’s ad group’s VP Todd Teresi now reports directly to services chief Eddy Cue — an indication of the company’s ads business’ increased importance.

The news follows a recent report by Digiday which discovered, based on job listings, Apple appeared to be developing its own demand-side platform. It wasn’t clear whether this DSP would be devoted to serving ads on the App Store or other Apple-owned apps, or if it had broader ambitions. Apple didn’t offer a comment on the report at the time.

Apple’s ability to carve out more of a slice of the ads market for itself comes after the company made changes to its iOS platform that damaged third-party app makers’ ability to target their users with personalized ads. Apple’s ATT platform (App Tracking Transparency) lets users opt out of being tracked — something Apple characterizes as only a user privacy feature. But the feature is also of great benefit to Apple, as it’s able to leverage the data it collects across its own first-party apps to offer personalized ads. In time, advertisers could shift some of their budgets to Apple’s personalized ads — particularly if their ad dollars spent on third-party apps proved to deliver less powerful results due to ATT’s impact.

If Apple were to grow its advertising footprint, it could also increase pressure on the company to be more transparent about the revenues generated by the individual entities within Apple’s Services business, which includes the App Store and its advertising arm. Today, Apple’s Services drive nearly a third of the company’s gross profit, a Bloomberg op-ed noted, stressing that investors should have a right to know how well these businesses are performing.

Tom Brady’s buzzy celebrity NFT startup Autograph banks $170M from Silicon Valley’s top crypto investors

Autograph, an NFT agency co-founded by athlete Tom Brady with a particularly deep bench of star power, has banked new funding from crypto investors who hope the platform can bring a new generation of celebrities and their fans into the fold of crypto collectibles.

The startup tells TechCrunch it has closed a $170 million Series B round led by Andreessen Horowitz and Kleiner Perkins with participation from Katie Haun’s new fund and Lightspeed partner Nicole Quinn. This comes on the heels of a Series A round co-led by 01A and Velvet Sea Ventures that closed this past July. The raise adds three new members to the board, including Haun, a16z’s Arianna Simpson and Kleiner Perkins’s Ilya Fushman.

They join an expansive board with some major names including Brady, Apple’s Eddy Cue, FTX’s Sam Bankman-Fried and artist Abel Tesfaye, also known as The Weeknd.

Autograph is essentially an agency go-between for celebrity athletes and entertainers looking to find a personal presence in the incredibly noisy NFT world. Celebrity NFT drops have been a mixed bag since the crypto collectibles skyrocketed to popularity early last year. While some have come into the space with thoughtful projects that reward their fans, there have been a multitude of ridicule-generating cash grabs.

Crypto has generally been a tough place for discerning celebrities to make a dent or earn a buck without losing some of their cultural capital. Matt Damon received a fair share of ridicule this month for appearing in a corny crypto ad campaign. Earlier this week Kim Kardashian and Floyd Mayweather were named in lawsuit brought by disgruntled investors who had backed a token that the two had earlier endorsed.

While fellow NFT unicorn Dapper Labs has inked partnerships with a handful overarching American sports league players associations including the NFL Players Association and the NBA’s Player Association, Autograph seems to be more focused on the individual athletes and the cult of personality built around them outside the context of the teams they play on or leagues they are part of. Autograph has a handful of early partners largely in the sports world, but the names represent some of the most famous athletes in their respective sports including Brady, Tiger Woods, Simone Biles, Derek Jeter, Naomi Osaka, Usain Bolt, Wayne Gretzky and Tony Hawk.

Autograph’s aim seems to be in building out an onramp to the space where triple-A listers can dip their toes into crypto influencing in a more highly curated environment.

In July, Autograph announced a partnership with DraftKings, though the startup tells TechCrunch it has been working with a number of marketplaces. Autograph has been primarily serving up its NFTs on the Polygon blockchain, which taps into Ethereum’s infrastructure but uses significantly less energy per transaction, a fact that’s likely important to celebrities who may be wary to open themselves up to environmental critiques of blockchain technologies.

Outside the sports world, Autograph has done drops with The Weeknd, Slam Magazine and officially-licensed NFTs for the horror series Saw.

Apple lowers commissions on in-app purchases for news publishers who participate in Apple News

Apple today is launching a new program that will allow subscription news organizations that participate in the Apple News app and meet certain requirements to lower their commission rate to 15% on qualifying in-app purchases taking place inside their apps on the App Store. Typically, Apple’s model for subscription-based apps involves a standard 30% commission during their first year on the App Store which then drops to 15% in year two. But the new Apple News Partner Program, announced today, will now make 15% the commission rate for participants starting on day one.

There are a few caveats to this condition, and they benefit Apple. To qualify, the news publisher must maintain a presence on Apple News and they have to provide their content in the Apple News Format (ANF). The latter is the JavaScript Object Notation (JSON) format that’s used to create articles for Apple News which are optimized for Mac, iPhone and other Apple mobile devices. Typically, this involves a bit of setup to translate news articles from a publisher’s website or from their CMS (content management system) to the supported JSON format. For WordPress and other popular CMS’s, there are also plugins available to make this process easier.

Meanwhile, for publishers headquartered outside one of the four existing Apple News markets — the U.S., U.K., Australia, or Canada — they can instead satisfy the program’s obligations by providing Apple with an RSS feed.

On the App Store, the partner app qualifying for the 15% commission must be used to deliver “original, professionally authored” news content, and they must offer their auto-renewable subscriptions using Apple’s in-app purchase system.

Image Credits: Apple

While there is some initial work involved in establishing the publisher’s connection to Apple News, it’s worth noting that most major publishers already participate on Apple’s platform. That means they won’t have to do any additional work beyond what they’re already doing in order to transition over to the reduced commission for their apps. However, the program also serves as a way to push news organizations to continue to participate in the Apple News ecosystem, as it will make more financial sense to do so across their broader business.

That will likely be an area of contention for publishers, who would probably prefer that the reduced App Store commission didn’t come with strings attached.

Some publishers already worry that they’re giving up too much control over their business by tying themselves to the Apple News ecosystem. Last year, for example, The New York Times announced it would exit its partnership with Apple News, saying that Apple didn’t allow it to have as direct a relationship with readers as it wanted, and it would rather drive readers to its own app and website.

Apple, however, would argue that it doesn’t stand in the way of publishers’ businesses — it lets them paywall their content and keep 100% of the ad revenue from the ads they sell. (If they can’t sell it all or would prefer Apple to do so on their behalf, they then split the commission with Apple, keeping 70% of revenues instead.) In addition, for the company’s Apple News+ subscription service — where the subscription revenue split is much higher — it could be argued that it’s “found money.” That is, Apple markets the service to customers the publisher hadn’t been able to attract on its own anyway.

The launch of the new Apple News Partner program comes amid regulatory scrutiny over how Apple manages its App Store business and more recently, proposed legislation aiming to address alleged anticompetitive issues both in the U.S. and in major App Store markets, like South Korea.

Sensing this shift in the market, Apple had already been working to provide itself cover from antitrust complaints and lawsuits — like the one underway now with Epic Games — by adjusting its App Store commissions. Last year, it launched the App Store Small Business Program, which also lowered commissions on in-app purchases from 30% to 15% — but only for developers earning up to $1 million in revenues.

This program may have helped smaller publishers, but it was clear some major publishers still weren’t satisfied. After the reduced commissions for small businesses were announced in November, the publisher trade organization Digital Content Next (DCN) — a representative for the AP, The New York Times, NPR, ESPN, Vox, The Washington Post, Meredith, Bloomberg, NBCU, The Financial Times, and others — joined the advocacy group and lobbying organization the Coalition for App Fairness (CAF) the very next month.

These publishers, who had previously written to Apple CEO Tim Cook to demand lower commissions — had other complaints about the revenue share beyond just the size of the split. They also didn’t want to be required to use Apple’s services for in-app purchases for their subscriptions, saying this “Apple tax” forces them to raise their prices for consumers.

It remains to be seen how these publishers will now react to the launch of the Apple News Partner program.

While it gives them a way to lower their App Store fees, it doesn’t address their broader complaints against Apple’s platform and its rules. If anything, it ties the lower fees to a program that locks them in further to the Apple ecosystem.

Apple, in a gesture of goodwill, also said today it would recommit support to three leading media non-profits, Common Sense Media, the News Literacy Project, and Osservatorio Permanente Giovani-Editori. These non-profits offer nonpartisan, independent media literacy programs, which Apple views as key to its larger mission to empower people to become smart and active news readers. Apple also said it would later announce further media literacy projects from other organizations. The company would not disclose the size of its commitment from a financial standpoint however, or discuss how much it has sent such organizations in the past.

“Providing Apple News customers with access to trusted information from our publishing partners has been our priority from day one,” said Eddy Cue, Apple’s senior vice president of Services, in a statement. “For more than a decade, Apple has offered our customers many ways to access and enjoy news content across our products and services. We have hundreds of news apps from dozens of countries around the world available in the App Store, and created Apple News Format to offer publishers a tool to showcase their content and provide a great experience for millions of Apple News users,” he added.

More details about the program and the application form will be available at the News Partner Program website.

Apple invests $50M into music distributor UnitedMasters alongside A16z and Alphabet

Independent music distribution platform and tool factory UnitedMasters has raised a $50M series B round led by Apple. A16z and Alphabet are participating again in this raise. United Masters is also entering a strategic partnership with Apple alongside this investment. 

If you’re unfamiliar with UnitedMasters, it’s a distribution company launched in 2017 by Steve Stoute, a former Interscope and Sony Music executive. The focus of UnitedMasters is to provide artists with a direct pipeline to data around the way that fans are interacting with their content and community, allowing them to connect more directly to offer tickets, merchandise and other commercial efforts. UnitedMasters also generally allows artists to retain control of their own masters.

Neither of these conditions are at all typical in the music industry. In a typical artist deal, recording companies retain all audience and targeting data as well as masters. This limits an artist’s ability to be agile, taking advantage of new technologies to foster a community. 

While Apple does invest in various companies, it typically does so out of its Advanced Manufacturing Fund to promote US manufacturing or strategically in partners that make critical components of its hardware like silicon foundries or glass manufacturing. Apple does a lot more purchasing than investing, typically, buying a company every few weeks or so to supplement one product effort or another. UnitedMasters, then, would be a relatively unique partnership, especially in the music space. 

I spoke to UnitedMasters CEO Steve Stoute about the deal and what it means for the businesses 1M current artists and new ones. Stoute credits Apple executive Eddy Cue having a philosophy aligned with the UnitedMasters vision with getting this deal done. 

“We want all artists to have the same opportunity,” says Stoute. “Currently, independent artists have less opportunity for success and we’re trying to remove that stigma.”

This infusion, Stoute says, will be used to hire talent that are mission oriented to take UnitedMasters global. They’re seeking local technical talent and artists talent to build out the platform worldwide. 

“Every artist needs access to a CTO,” Stoute says. “Some of the value of what a manager is today for an artist needs to be transferred to that role.”

UnitedMasters wants to provide that technical edge at scale, allowing artists to build out their fanbase at a community level.

Currently, UnitedMasters has deals with the NBA, ESPN, TikTok, Twitch and others that allow artists to tap big brand deals that would normally be brokered by a label and manager. It also has a direct distribution app that allows publishing to all of the major streaming services. Most importantly, they can check stream, fan and earnings data at a glance. 

“Steve Stoute and UnitedMasters provide creators with more opportunities to advance their careers and bring their music to the world,” said Apple’s Eddy Cue in a release statement. “The contributions of independent artists play a significant role in driving the continued growth and success of the music industry, and UnitedMasters, like Apple, is committed to empowering creators.”

“UnitedMasters has completely transformed the way artists create, retain ownership in their work, and connect with their fans,” said Ben Horowitz, Co-Founder and General Partner of Andreessen Horowitz in a release. “We are excited to work with Steve and team to build a better, bigger, and far more profitable world for musical artists.” 

We are currently at an inflection point in the way that artists and fans connect with one another. Though there have been seemingly endless ways for artists to get their messages out or speak to fans using social media and other platforms, the actual business of distributing work to a community and making money from that work has been out of their hands completely since the beginning of the recording industry. Recent developments like NFTs, DAOs and social tokens, as well as an explosion of DTC frameworks have begun to re-write that deal. But the major players have yet to make the truly aggressive strides they need to in order to embrace this ‘artist centric’ new world. 

The mechanics of distribution have been based on a framework defined by DRM and the DMCA for decades. This framework was always marketed as a way to protect value for the artist but was in fact architected to protect value for the distributor. We need a rethinking of the entire distribution layer.

As I mentioned when reporting the UnitedMasters + TikTok deal, it’s going to be instrumental in a more equitable future for artists:

It’s beyond time for the creators of The Culture to benefit from that culture. That’s why I find this UnitedMasters deal so interesting. Offering a direct pipeline to audiences without the attendant vulture-ism of the recording industry apparatus is really well-aligned with a platform like TikTok, which encourages and enables “viral sounds” with collaborative performances. Traditional deal structures are not well-suited to capturing viral hype, which can rise and fall within weeks without additional fuel.

In music, Apple is at the center of this maelstrom along with a few other major players like Spotify. One of the big misses in recent years for Apple Music, in my opinion, was Apple’s failure to turn Apple Music Connect into an industry-standard portal that allowed artists to connect broadly with fans, distribute directly, sell tickets and merchandise but — most importantly — to foster and own their community. 

A UnitedMasters tie up isn’t a straight line to that goal, but it’s definitely got the ingredients. I’m looking forward to seeing what this produces. 

Image Credits: Steve Stoute

DuckDuckGo debuts map search results using Apple Maps

DuckDuckGo has a new, unlikely partner in search: Apple.

The privacy-focused search engine that promises to never track its users said Tuesday it’s now using data provided by Apple Maps to power its map-based search results. Although DuckDuckGo had provided limited mapping results for a while using data from open-source service OpenStreetMap, it never scaled its features to those of its search engine rivals, notably Google and Bing.

Now, DuckDuckGo will return addresses, businesses, geographical locations, and nearby places using Apple Maps by default. (When we tested, directions and transit times open up in Apple Maps on your Mac, iPhone, or iPad — but on non-Apple devices, the directions defaults to and opens in Bing.)

In using Apple’s mapping data, DuckDuckGo will become one of the biggest users of Apple Maps to date, six months after Apple said it would open up Apple Maps, long only available on Macs, iPhones and iPads, to the web.

“We’re excited to work closely with Apple to set a new standard of trust online, and we hope you’ll enjoy this update,” said the search engine in a blog post.

DuckDuckGo in the Tor browser, using the new Apple Maps feature. (Screenshot: TechCrunch)

In reality, the partnership isn’t that unsurprising at all.

Apple faced flak for ditching Google Maps in iOS and rushing its overhauled Maps service out to the market, prompting a rare mea culpa from chief executive Tim Cook, apologizing for the disastrous rollout. At its most recent Worldwide Developers Conference in June, Apple promised a do-over, offering reliability and stability — but more importantly, privacy.

Where Google tracks everything you do, where you go and what you search for, Apple has long said it doesn’t want to know. Any data that Apple collects is anonymous, said Eddy Cue, Apple internet software and services chief, in an interview with TechCrunch last year. “We specifically don’t collect data, even from point A to point B,” said Cue. By anonymizing the data, Apple doesn’t know where you came from or where you went, or even who took the trip.

DuckDuckGo finally brings a much-needed feature to the search engine, while keeping true to its privacy-focused roots as a non-tracking search rival to Google.

“At DuckDuckGo, we believe getting the privacy you deserve online should be as simple as closing the blinds,” the company said. “Naturally, our strict privacy policy of not collecting or sharing any personal information extends to this integration.”

“You are still anonymous when you perform map and address-related searches on DuckDuckGo,” the search engine said.

In a separate note, DuckDuckGo said users can turn on their location for better “nearby” search results, but promises to not store the data or use it for any purposes. “Even if you opt-in to sharing a more accurate location, your searches will still be completely anonymous,” said DuckDuckGo.

“We do not send any personally identifiable information such as IP address to Apple or other third parties,” the company said.

DuckDuckGo processes 30 million daily searches, up by more than 50 percent year-over-year, the company said last year.

Apple is rebuilding Maps from the ground up

I’m not sure if you’re aware, but the launch of Apple Maps went poorly. After a rough first impression, an apology from the CEO, several years of patching holes with data partnerships and some glimmers of light with long-awaited transit directions and improvements in business, parking and place data, Apple Maps is still not where it needs to be to be considered a world class service.

Maps needs fixing.

Apple, it turns out, is aware of this, so It’s re-building the maps part of Maps.

It’s doing this by using first-party data gathered by iPhones with a privacy-first methodology and its own fleet of cars packed with sensors and cameras. The new product will launch in San Francisco and the Bay Area with the next iOS 12 Beta and will cover Northern California by fall.

Every version of iOS will get the updated maps eventually and they will be more responsive to changes in roadways and construction, more visually rich depending on the specific context they’re viewed in and feature more detailed ground cover, foliage, pools, pedestrian pathways and more.

This is nothing less than a full re-set of Maps and it’s been 4 years in the making, which is when Apple began to develop its new data gathering systems. Eventually, Apple will no longer rely on third-party data to provide the basis for its maps, which has been one of its major pitfalls from the beginning.

“Since we introduced this six years ago — we won’t rehash all the issues we’ve had when we introduced it — we’ve done a huge investment in getting the map up to par,” says Apple SVP Eddy Cue, who now owns Maps in an interview last week.  “When we launched, a lot of it was all about directions and getting to a certain place. Finding the place and getting directions to that place. We’ve done a huge investment of making millions of changes, adding millions of locations, updating the map and changing the map more frequently. All of those things over the past six years.”

But, Cue says, Apple has room to improve on the quality of Maps, something that most users would agree on, even with recent advancements.

“We wanted to take this to the next level,” says Cue. “We have been working on trying to create what we hope is going to be the best map app in the world, taking it to the next step. That is building all of our own map data from the ground up.”

In addition to Cue, I spoke to Apple VP Patrice Gautier and over a dozen Apple Maps team members at its mapping headquarters in California this week about its efforts to re-build Maps, and to do it in a way that aligned with Apple’s very public stance on user privacy.

If, like me, you’re wondering whether Apple thought of building its own maps from scratch before it launched Maps, the answer is yes. At the time, there was a choice to be made about whether or not it wanted to be in the business of Maps at all. Given that the future of mobile devices was becoming very clear, it knew that mapping would be at the core of nearly every aspect of its devices from photos to directions to location services provided to apps. Decision made, Apple plowed ahead, building a product that relied on a patchwork of data from partners like TomTom, OpenStreetMap and other geo data brokers. The result was underwhelming.

Almost immediately after Apple launched Maps, it realized that it was going to need help and it signed on a bunch of additional data providers to fill the gaps in location, base map, point-of-interest and business data.

It wasn’t enough.

“We decided to do this just over four years ago. We said, “Where do we want to take Maps? What are the things that we want to do in Maps? We realized that, given what we wanted to do and where we wanted to take it, we needed to do this ourselves,” says Cue.

Because Maps are so core to so many functions, success wasn’t tied to just one function. Maps needed to be great at transit, driving and walking — but also as a utility used by apps for location services and other functions.

Cue says that Apple needed to own all of the data that goes into making a map, and to control it from a quality as well as a privacy perspective.

There’s also the matter of corrections, updates and changes entering a long loop of submission to validation to update when you’re dealing with external partners. The Maps team would have to be able to correct roads, pathways and other updating features in days or less, not months. Not to mention the potential competitive advantages it could gain from building and updating traffic data from hundreds of millions of iPhones, rather than relying on partner data.

Cue points to the proliferation of devices running iOS, now numbering in the millions, as a deciding factor to shift its process.

“We felt like because the shift to devices had happened — building a map today in the way that we were traditionally doing it, the way that it was being done — we could improve things significantly, and improve them in different ways,” he says. “One is more accuracy. Two is being able to update the map faster based on the data and the things that we’re seeing, as opposed to driving again or getting the information where the customer’s proactively telling us. What if we could actually see it before all of those things?”

I query him on the rapidity of Maps updates, and whether this new map philosophy means faster changes for users.

“The truth is that Maps needs to be [updated more], and even are today,” says Cue. “We’ll be doing this even more with our new maps, [with] the ability to change the map real-time and often. We do that every day today. This is expanding us to allow us to do it across everything in the map. Today, there’s certain things that take longer to change.

“For example, a road network is something that takes a much longer time to change currently. In the new map infrastructure, we can change that relatively quickly. If a new road opens up, immediately we can see that and make that change very, very quickly around it. It’s much, much more rapid to do changes in the new map environment.”

So a new effort was created to begin generating its own base maps, the very lowest building block of any really good mapping system. After that, Apple would begin layering on living location data, high resolution satellite imagery and brand new intensely high resolution image data gathered from its ground cars until it had what it felt was a ‘best in class’ mapping product.

There is only really one big company on earth who owns an entire map stack from the ground up: Google .

Apple knew it needed to be the other one. Enter the vans.

Apple vans spotted

Though the overall project started earlier, the first glimpse most folks had of Apple’s renewed efforts to build the best Maps product was the vans that started appearing on the roads in 2015 with ‘Apple Maps’ signs on the side. Capped with sensors and cameras, these vans popped up in various cities and sparked rampant discussion and speculation.

The new Apple Maps will be the first time the data collected by these vans is actually used to construct and inform its maps. This is their coming out party.

Some people have commented that Apple’s rigs look more robust than the simple GPS + Camera arrangements on other mapping vehicles — going so far as to say they look more along the lines of something that could be used in autonomous vehicle training.

Apple isn’t commenting on autonomous vehicles, but there’s a reason the arrays look more advanced: they are.

Earlier this week I took a ride in one of the vans as it ran a sample route to gather the kind of data that would go into building the new maps. Here’s what’s inside.

In addition to a beefed up GPS rig on the roof, four LiDAR arrays mounted at the corners and 8 cameras shooting overlapping high-resolution images – there’s also the standard physical measuring tool attached to a rear wheel that allows for precise tracking of distance and image capture. In the rear there is a surprising lack of bulky equipment. Instead, it’s a straightforward Mac Pro bolted to the floor, attached to an array of solid state drives for storage. A single USB cable routes up to the dashboard where the actual mapping capture software runs on an iPad.

While mapping, a driver…drives, while an operator takes care of the route, ensuring that a coverage area that has been assigned is fully driven and monitoring image capture. Each drive captures thousands of images as well as a full point cloud (a 3D map of space defined by dots that represent surfaces) and GPS data. I later got to view the raw data presented in 3D and it absolutely looks like the quality of data you would need to begin training autonomous vehicles.

More on why Apple needs this level of data detail later.

When the images and data are captured, they are then encrypted on the fly immediately and recorded on to the SSDs. Once full, the SSDs are pulled out, replaced and packed into a case which is delivered to Apple’s data center where a suite of software eliminates private information like faces, license plates and other info from the images. From the moment of capture to the moment they’re sanitized, they are encrypted with one key in the van and the other key in the data center. Technicians and software that are part of its mapping efforts down the pipeline from there never see unsanitized data.

This is just one element of Apple’s focus on the privacy of the data it is utilizing in New Maps.

Probe data and Privacy

Throughout every conversation I have with any member of the team throughout the day, privacy is brought up, emphasized. This is obviously by design as it wants to impress upon me as a journalist that it’s taking this very seriously indeed, but it doesn’t change the fact that it’s evidently built in from the ground up and I could not find a false note in any of the technical claims or the conversations I had.

Indeed, from the data security folks to the people whose job it is to actually make the maps work well, the constant refrain is that Apple does not feel that it is being held back in any way by not hoovering every piece of customer-rich data it can, storing and parsing it.

The consistent message is that the team feels it can deliver a high quality navigation, location and mapping product without the directly personal data used by other platforms.

“We specifically don’t collect data, even from point A to point B,” notes Cue. “We collect data — when we do it —in an anonymous fashion, in subsections of the whole, so we couldn’t even say that there is a person that went from point A to point B. We’re collecting the segments of it. As you can imagine, that’s always been a key part of doing this. Honestly, we don’t think it buys us anything [to collect more]. We’re not losing any features or capabilities by doing this.”

The segments that he is referring to are sliced out of any given person’s navigation session. Neither the beginning or the end of any trip is ever transmitted to Apple. Rotating identifiers, not personal information, are assigned to any data or requests sent to Apple and it augments the ‘ground truth’ data provided by its own mapping vehicles with this ‘probe data’ sent back from iPhones.

Because only random segments of any person’s drive is ever sent and that data is completely anonymized, there is never a way to tell if any trip was ever a single individual. The local system signs the IDs and only it knows who that ID refers to. Apple is working very hard here to not know anything about its users. This kind of privacy can’t be added on at the end, it has to be woven in at the ground level.

Because Apple’s business model does not rely on it serving, say, an ad for a Chevron on your route to you, it doesn’t need to even tie advertising identifiers to users.

Any personalization or Siri requests are all handled on-board by the iOS device’s processor. So if you get a drive notification that tells you it’s time to leave for your commute, that’s learned, remembered and delivered locally, not from Apple’s servers.

That’s not new, but it’s important to note given the new thing to take away here: Apple is flipping on the power of having millions of iPhones passively and actively improving their mapping data in real time.

In short: traffic, real-time road conditions, road systems, new construction and changes in pedestrian walkways are about to get a lot better in Apple Maps.

The secret sauce here is what Apple calls probe data. Essentially little slices of vector data that represent direction and speed transmitted back to Apple completely anonymized with no way to tie it to a specific user or even any given trip. It’s reaching in and sipping a tiny amount of data from millions of users instead, giving it a holistic, real-time picture without compromising user privacy.

If you’re driving, walking or cycling, your iPhone can already tell this. Now if it knows you’re driving it can also send relevant traffic and routing data in these anonymous slivers to improve the entire service. This only happens if your maps app has been active, say you check the map, look for directions etc. If you’re actively using your GPS for walking or driving, then the updates are more precise and can help with walking improvements like charting new pedestrian paths through parks — building out the map’s overall quality.

All of this, of course, is governed by whether you opted into location services and can be toggled off using the maps location toggle in the Privacy section of settings.

Apple says that this will have a near zero effect on battery life or data usage, because you’re already using the ‘maps’ features when any probe data is shared and it’s a fraction of what power is being drawn by those activities.

From the point cloud on up

But maps cannot live on ground truth and mobile data alone. Apple is also gathering new high resolution satellite data to combine with its ground truth data for a solid base map. It’s then layering satellite imagery on top of that to better determine foliage, pathways, sports facilities, building shapes and pathways.

After the downstream data has been cleaned up of license plates and faces, it gets run through a bunch of computer vision programming to pull out addresses, street signs and other points of interest. These are cross referenced to publicly available data like addresses held by the city and new construction of neighborhoods or roadways that comes from city planning departments.

But one of the special sauce bits that Apple is adding to the mix of mapping tools is a full on point cloud that maps the world around the mapping van in 3D. This allows them all kinds of opportunities to better understand what items are street signs (retro-reflective rectangular object about 15 feet off the ground? Probably a street sign) or stop signs or speed limit signs.

It seems like it could also enable positioning of navigation arrows in 3D space for AR navigation, but Apple declined to comment on ‘any future plans’ for such things.

Apple also uses semantic segmentation and Deep Lambertian Networks to analyze the point cloud coupled with the image data captured by the car and from high-resolution satellites in sync. This allows 3D identification of objects, signs, lanes of traffic and buildings and separation into categories that can be highlighted for easy discovery.

The coupling of high resolution image data from car and satellite, plus a 3D point cloud results in Apple now being able to produce full orthogonal reconstructions of city streets with textures in place. This is massively higher resolution and easier to see, visually. And it’s synchronized with the ‘panoramic’ images from the car, the satellite view and the raw data. These techniques are used in self driving applications because they provide a really holistic view of what’s going on around the car. But the ortho view can do even more for human viewers of the data by allowing them to ‘see’ through brush or tree cover that would normally obscure roads, buildings and addresses.

This is hugely important when it comes to the next step in Apple’s battle for supremely accurate and useful Maps: human editors.

Apple has had a team of tool builders working specifically on a toolkit that can be used by human editors to vet and parse data, street by street. The editor’s suite includes tools that allow human editors to assign specific geometries to flyover buildings (think Salesforce tower’s unique ridged dome) that allow them to be instantly recognizable. It lets editors look at real images of street signs shot by the car right next to 3D reconstructions of the scene and computer vision detection of the same signs, instantly recognizing them as accurate or not.

Another tool corrects addresses, letting an editor quickly move an address to the center of a building, determine whether they’re misplaced and shift them around. It also allows for access points to be set, making Apple Maps smarter about the ‘last 50 feet’ of your journey. You’ve made it to the building, but what street is the entrance actually on? And how do you get into the driveway? With a couple of clicks, an editor can make that permanently visible.

“When we take you to a business and that business exists, we think the precision of where we’re taking you to, from being in the right building,” says Cue. “When you look at places like San Francisco or big cities from that standpoint, you have addresses where the address name is a certain street, but really, the entrance in the building is on another street. They’ve done that because they want the better street name. Those are the kinds of things that our new Maps really is going to shine on. We’re going to make sure that we’re taking you to exactly the right place, not a place that might be really close by.”

Water, swimming pools (new to Maps entirely), sporting areas and vegetation are now more prominent and fleshed out thanks to new computer vision and satellite imagery applications. So Apple had to build editing tools for those as well.

Many hundreds of editors will be using these tools, in addition to the thousands of employees Apple already has working on maps, but the tools had to be built first, now that Apple is no longer relying on third parties to vet and correct issues.

And the team also had to build computer vision and machine learning tools that allow it to determine whether there are issues to be found at all.

Anonymous probe data from iPhones, visualized, looks like thousands of dots, ebbing and flowing across a web of streets and walkways, like a luminescent web of color. At first, chaos. Then, patterns emerge. A street opens for business, and nearby vessels pump orange blood into the new artery. A flag is triggered and an editor looks to see if a new road needs a name assigned.

A new intersection is added to the web and an editor is flagged to make sure that the left turn lanes connect correctly across the overlapping layers of directional traffic. This has the added benefit of massively improved lane guidance in the new Apple Maps.

Apple is counting on this combination of human and AI flagging to allow editors to first craft base maps and then also maintain them as the ever changing biomass wreaks havoc on roadways, addresses and the occasional park.

Here there be Helvetica

Apple’s new Maps, like many other digital maps, display vastly differently depending on scale. If you’re zoomed out, you get less detail. If you zoom in, you get more. But Apple has a team of cartographers on staff that work on more cultural, regional and artistic levels to ensure that its Maps are readable, recognizable and useful.

These teams have goals that are at once concrete and a bit out there — in the best traditions of Apple pursuits that intersect the technical with the artistic.

The maps need to be usable, but they also need to fulfill cognitive goals on cultural levels that go beyond what any given user might know they need. For instance, in the US, it is very common to have maps that have a relatively low level of detail even at a medium zoom. In Japan, however, the maps are absolutely packed with details at the same zoom, because that increased information density is what is expected by users.

This is the department of details. They’ve reconstructed replicas of hundreds of actual road signs to make sure that the shield on your navigation screen matches the one you’re seeing on the highway road sign. When it comes to public transport, Apple licensed all of the type faces that you see on your favorite subway systems, like Helvetica for NYC. And the line numbers are in the exact same order that you’re going to see them on the platform signs.

It’s all about reducing the cognitive load that it takes to translate the physical world you have to navigate through into the digital world represented by Maps.

Bottom line

The new version of Apple Maps will be in preview next week with just the Bay Area of California going live. It will be stitched seamlessly into the ‘current’ version of Maps, but the difference in quality level should be immediately visible based on what I’ve seen so far.

Better road networks, more pedestrian information, sports areas like baseball diamonds and basketball courts, more land cover including grass and trees represented on the map as well as buildings, building shapes and sizes that are more accurate. A map that feels more like the real world you’re actually traveling through.

Search is also being revamped to make sure that you get more relevant results (on the correct continents) than ever before. Navigation, especially pedestrian guidance, also gets a big boost. Parking areas and building details to get you the last few feet to your destination are included as well.

What you won’t see, for now, is a full visual redesign.

“You’re not going to see huge design changes on the maps,” says Cue. “We don’t want to combine those two things at the same time because it would cause a lot of confusion.”

Apple Maps is getting the long awaited attention it really deserves. By taking ownership of the project fully, Apple is committing itself to actually creating the map that users expected of it from the beginning. It’s been a lingering shadow on iPhones, especially, where alternatives like Google Maps have offered more robust feature sets that are so easy to compare against the native app but impossible to access at the deep system level.

The argument has been made ad nauseam, but it’s worth saying again that if Apple thinks that mapping is important enough to own, it should own it. And that’s what it’s trying to do now.

“We don’t think there’s anybody doing this level of work that we’re doing,” adds Cue. “We haven’t announced this. We haven’t told anybody about this. It’s one of those things that we’ve been able to keep pretty much a secret. Nobody really knows about it. We’re excited to get it out there. Over the next year, we’ll be rolling it out, section by section in the US.”

Apple Music now has 38 million paid members

 Here’s another tidbit from Eddy Cue’s content-packed SXSW appearance yesterday. Along with a dive into Apple’s video content strategy, the company’s SVP offered up some insight into the health of its music streaming service. Cue noted that Apple Music has hit 38 million paid subscribers — an increase of two million since the company reported last month. It’s… Read More