We tried out Canon’s VR calling app Kokomo

When you think Canon, you might think cameras, scanners, printers and such. At CES this year, the company did a stride toward the softer side of tech, taking a broader view of what imaging means. Today, the VR calling software the company showed off back in January is available to you — if you have a VR headset such as the popular Oculu… I mean Meta Quest 2.

In a limited preview earlier this month, I was able to try out Canon’s new VR calling software platform, which involves a phone and a VR headset. Overall and in theory, the system is super-well-thought-out: You scan your face using an app on your phone, which builds a model so the app can replace the part of your face that’s covered up by your VR mask. It also asks you how tall you are, so it can scale your picture proportionally to the person you are talking to — a nice touch; as a tall person it often feels unheimlich to be face-to-face with people when I’m in VR.

I knew we would do the demo in a desert setting so I wore a Hawaiian shirt and shorts. Nope, there’s no way of looking cool in VR — and the removing-the-mask-from-my-face feature worked about 20% of the time. Image Credits: Screenshot of the Kokomo / Canon app in VR

Once you’ve gone through the setup process, you set your phone at waist-height and step a few steps away, don a VR mask and dive into a conversation. The phone uses its front-facing camera to capture a live picture of you, while the VR headset shows your fellow caller. If everything goes to plan, it’s telepresence at its finest, without the vast expenditure of typical telepresence systems.

That’s the theory — and what Canon is working toward with its platform. I’m loving the vision, but the reality of the current system isn’t quite there yet.

In practice, things are not quite smooth. In my test call with the Kokomo team, I’m rather generous if I say I saw the other person in the call refresh at about 3-4 frames per second. That’s not enough to make the call feel smooth, and it rather got in the way of feeling the presence of the, er, telepresence. The avatar I was speaking to was also two-dimensional, which is something we haven’t seen in VR for a while — in effect, it feels like the person on the other side of the call is an animated cardboard cutout. That, combined with the low framerate and the haphazardness of the VR mask removal (it appeared and disappeared at regular intervals), did everything it could to ruin the immersion.

I can only assume that over time the VR mask removal tech will be less Picasso-esque. Image Credits: Screenshot of the Kokomo / Canon app in VR

The biggest praise I can heap on Kokomo is that despite being early in its technical journey — and despite the litany of bugs and early-software gremlins — I think it shows tremendous promise. For now, the team tells TechCrunch it won’t be charging for the service; it wants to learn and get feedback from early users to help direct product development.

It’s hard to predict what happens next for the product; VR adoption is growing, and telepresence is a compelling use case for spending some time with your friends in VR. Right now, the tech is good enough to enable you to dream of the future that’s yet to come, but not quite there to the level that it makes sense. All of these things could change, and the most interesting takeaway from this product, I believe, is Canon’s dedication to bringing Kokomo to market and putting it in front of customers. It’s definitely one to keep an eye on — and, if you have the required hardware, worth trying out with a friend.

We tried out Canon’s VR calling app Kokomo by Haje Jan Kamps originally published on TechCrunch

Sonic identity is the missing component for social metaverse experiences

In the many conversations around “the metaverse” sparked by the Facebook corporate name transition to Meta, much has focused on the visual elements. What’s hardly mentioned is audio. Yet voice really matters when making a virtual environment come to life.

Sometimes, it’s everything.

Just ask Spike Jonze. The movie director discarded the original voice actor in the title role of his 2013 film “Her” and substituted Scarlett Johansson’s sultry timbres. Although Samantha, a computer operating system, never appeared in the flesh, Jonze felt the original actress hadn’t nailed the emotion required to create a three-dimensional persona.

Voice was critical to creating a fleshed-out character that could absorb the viewer into the story’s premise and make it fully believable.

As The Washington Post noted, many keystones of the Meta vision of the metaverse already exist in video gaming — only in disconnected gaming worlds. And in the gaming universe, voice is playing an increasingly important role. Meta promises a unified, interoperable experience, but without a rich panoply of highly textured, lifelike digital voices, the metaverse will be incomplete rather than inclusive and immersive.

The McGurk Effect research of the mid-1970s observed the cognitive dissonance resulting from mismatched audio and visual perceptions; voices that don’t fully mesh with an avatar can rip the participant from the virtual environment.

Expressing the real you

Humans are social beings, and the metaverse as currently promoted is a social environment where participants create unique personas in both home and workplace settings. Avatars will allow players to express themselves the way they want to be seen — as human, alien, animal, vegetable, cartoon or myriad other options. Players can try on new “looks” temporarily, the way they try new outfits. Gender and species are fluid.

Changing identity is hamstrung, however, if people are not able to change how they sound along with their visual presence. Having your voice match the persona presented to other people is a core element of a personalized player identity. It is a situation many people are already accustomed to from video games.

If you encounter a gritty, bearded, hulking knight in a game you are playing, you would expect that character to have a deep, gruff voice, accompanied by the clank of armor. Game companies make sure to deliver this, with non-player characters (NPCs) being carefully crafted by voice actors and audio specialists to provide an immersive experience.

Yet in online gaming environments or in the future metaverse, where the knight is the representation of an actual person, you will have a vastly different experience. You may be startled to hear a high-pitched teenager with bad microphone quality instead of the anticipated gravelly, mature voice. The drastic incongruence between sound and vision shreds the immersive quality of the experience. Metaverse avatars can only be fully immersive if they allow people to create full digital experiences.

Providing cover

In addition to enabling immersion, sonic identity technology can also allow players to slip into “true” pseudonymity. They can fully become the person (or being) they want others to see — which for many people is powerful protection from a sometimes hostile online environment. It can disguise a geographical accent so the participant can more smoothly integrate a player community (a capability an offshore customer support call center might benefit from). For people with vocal tics, it can cloak a physical impairment they’d rather not reveal.

Voice changing technology can also help to mitigate online discrimination and harassment. A research study published in the International Journal of Mental Health and Addiction in 2019 notes that female gamers frequently avoid verbal communication with other players to reduce unpleasant interactions. Voice changing technology can allow them to participate in fully pseudonymous conversations, free of a specified gender, in which they might feel more comfortable in expressing themselves.

Regardless of the “why,” researchers in the scholarly journal Human-Computer Interaction concluded in 2014 that “voice radically transforms the experience of online gaming, making virtual spaces more intensely social.”

From my own internal company data, it’s clear that players who communicate with voice alter egos feel more engrossed in the game, engage it for longer periods of time and spend more money within the game as a result.

What’s missing in the metaverse

A truly complete immersive experience requires a combination of 3D visuals and real-time audio to enable people to express themselves in the way they want to be heard. Participants want a sonic representation of themselves that is just as original and unique as their visual avatar — and they want the tools to customize their voice as meticulously as their appearance. There must be a harmonious marriage of both augmented audio and 3D video to keep the player immersed and engaged.

Real-time audio defines how people can bring the ultimate individuality to their content, making audio the great equalizer of the metaverse. Unfortunately, the current voice experience is challenged to offer the type of immersive qualities that will live up to the promise of an all-encompassing metaverse.

Real-time audio personas are still restrictive at best, despite experimentation by dogged early adopters. The tools to shape a person’s voice to match their digital self are limited and sound quality does not yet match visual quality.

Yet recent advancements in available audio technology are making it far easier for players to create a unique sonic identity. New solutions available to platform and game developers enable writers, producers and audio engineers to incorporate voice modification technology within their games to produce natural-sounding and fantasy voices on demand, in real time.

It offers the potential to generate new avenues for monetization by delivering an inclusive and immersive auditory experience that lures players and keeps them fully focused and engaged in the experience rather than dropping away.

Companies are investing in powerful tools that enable people to shape the visual representation of themselves in a digital space. They must not overlook the customized sonic identity for a matching social audio experience that makes the digital representation seamless.

The metaverse won’t be complete without it.

Our new hybrid lives: Tactile virtual experiences and hardware that lives with us

With hybrid models taking off across many aspects of society, it’s clear that though they offer incredible flexibility, the boundary lines between work and personal life are becoming increasingly blurred and emotionally draining.

Ritual has always been a powerful force in shaping our mental and emotional states; the gathering of people, physical totems, wardrobe and space design all work to choreograph that experience. But for people in the hybrid workforce, many of the rituals to which they’ve become accustomed are no longer accessible—their daily work experience involves no gathering, no change in location, and little (if any) wardrobe change.

We are doubling down on hybrid virtual experiences, even though studies reveal that young people who spend more than seven hours a day staring at a screen are more susceptible to depression, anxiety and have greater difficulty in completing tasks. Furthermore, employees are reporting fatigue and exhaustion from a sea of back-to-back meetings that stretch across multiple time zones, making the days feel endless.

Given that so much of the population is currently reliant on computing devices to engage in everything from work and school to shopping, banking and healthcare, we have to start taking a harder look at how we’re designing and developing those devices to better equip us for new rituals for the hybrid virtual world.

Today, computing devices account for every possible scenario, from the traditional desktop workstation to the ultra-portable handheld mobile phone. But what if the design of these objects could help users enforce the boundaries between work and personal life?

For instance, a device with a keyboard in front of a screen conveys “productivity tool,” while a touch tablet experience feels more casual and entertainment-focused. What if remote workers could have the option to switch between these two modalities to signal a switch from “work” to “personal”?

Another area that has exploded into the tech spotlight is video chat and conferencing tools. For many of us, the majority of our interactions are now playing out via virtual meetings on video conferencing apps. HD webcams and ring lights have been in high demand, and the number of virtual backgrounds and effects multiply daily.

But there are still many challenges and limitations to the video conference experience, partly because it’s so dependent on the hardware design. Tools like Zoom, Google Hangouts and Teams have all been racing to keep up with the latest upgrades, but the software can only go so far without tackling hardware hurdles like integrated lighting sources, improved audio or even tactile feedback.

However, if we start to accept these paradigm shifts of in-person to virtual, we can begin to design for the future normal with hardware upgrades like a camera lens no larger than a pixel that disappears into the screen to make it appear as if users are making direct eye contact with their colleagues. Other areas, such as the application of temperature and tactile technologies, can help us feel deeper connections with one another via virtual spaces. There may also be new possibilities in exploring olfactory technologies as immersive experiences continue to evolve.

But what does this hardware evolution actually look like when it comes to production and consumption? While the expediency and convenience of technology is certainly impressive, it comes at a cost to our planet.

Have consumers become Earth’s abusers?

When I think about my most cherished possessions, what they have in common is that they are old and rare. Of course, this is typical of valuable items, but why couldn’t we bring this value system to our tech products? While I swap out my iPhone every year or two, I take tremendous joy in upgrading parts on my Ducati motorcycle bit by bit. I would never think of tossing it out for a brand new one.

As consumer demand for sustainable solutions increase, hardware companies must adjust their offerings. Powerful brands like Apple could be a great leader in strong regenerative practices. Building your own desktop PC is nothing new (especially for hardcore gamers) but imagine a future where all portable tech is modular with swappable upgrades. What if 50 years from now, your smartphone from 2025 is a still functional and highly valued piece of vintage tech?

The reality of our new normal is that the plethora of devices is not going away, while software developments are continuing to make leaps and bounds. It’s time we started thinking about our devices as objects to keep and care for, repairing and refurbishing things like phones and computers to keep up with the latest advancements, much like we do with our cars or even our homes.

Birmingham-based Help Lightning raises $8 million for its remote training and support tools

In the four years since Help Lightning first began pitching its services out of its Birmingham, Ala. headquarters, the company has managed to sign up 100 customers including some large Fortune 500 companies like Cox Communications, Siemens, and Boston Scientific.

Now, with an additional $8 million in financing from Resolve Growth Partners, the company is hoping to expand its sales and marketing efforts and continue to refine its product.

The technology was initially invented by Bart Guthrie, a neurosurgeon at the University of Alabama at Birmingham, who wanted a way to improve telepresence technologies so he could assist with remote surgeries.

What Guthrie developed was a technology that could merge video streams to that experts could remotely monitor, manage, and assist in everything from service repairs to surgery.

“Think of it as a video call on steroids,” says Gary York, the company’s chief executive officer. A serial entrepreneur, York was brought on board by Guthrie to help commercialize the technology four years ago.

The technology works on any android or iOS device and is accessed through a mobile browser. The company now boasts over 100 customers including Cox, Canon, Unisys, and Boston Scientific. And its usage has soared since the advent of the pandemic, according to York.

“We saw call volume quadruple,” he said.

For instance, Cox Communications uses the technology to provide virtual trouble shooting to replace in-home service visits for customers. At Siemens, service technicians who fix medical imaging and lab diagnostic equipment can use the Help Lightning to link up with experts to troubleshoot fixes in real time. York would not comment on pricing, but said that the company provides custom quotes based on usage.

“After evaluating the virtual expertise software market for over a year, our diligence is clear that Help Lightning has built a highly differentiated solution that is valued by its customers” said Jit Sinha, co-founder and Managing Director from Resolve, in a statement earlier this week. “Help Lightning has a tremendous opportunity to power the success of this rapidly emerging market. We’re thrilled to be partnering with Gary York and his talented team.”

 

Smart TV hub Solaborate secures $10M Series A and a go-to-market partnership

When siblings Labinot and Mimoza Bytyqi fled the war in Kosovo in 1999, arriving as refugees on the West Coast of the US, they would have had no idea they’d go on to launch a technology company together.

But as adults, the pair set up attacking the $6.7 billion telepresence and video communication category which hasn’t evolved much since the older business systems form Cisco and Polycom . By integrating their Solaborate device with Smart TVs, the entrepreneurs have come up with a drastically cheaper device and platform.

Solaborate has now closed a $10 million Series A funding round from EPOS and Demant Group. EPOS is a newly established company under the healthcare tech company Demant Group in Denmark which makes high-end audio solutions designed for enterprise and gaming. The funding will be used to accelerate the development of Solaborate’s new product line of all-in-one HELLO devices and its cloud communication platform.

After two successful Kickstarter campaigns, Solaborate will now work with EPOS to combine compute, microphones, speakers and Smart TVs with their technology to create products fully-owned by and branded under EPOS. These will include Solaborate’s patented auto echo-cancellation delay.

Labinot Bytyqi, founder and CE) said: “We believe that privacy is a fundamental human right and that’s why we engineered HELLO devices with video and audio built-in hack-proof privacy controls and end-to-end encryption for everyone’s protection and peace of mind.”

A HELLO device require only two cables – HDMI and power – and then turns any TV into a voice-controlled open cross-platform communication and collaboration device supporting video conferencing platforms such as Microsoft Teams, Google Hangouts Meet, Zoom, Skype, Cisco WebEx, Facebook Messenger, WeChat, BlueJeans, Fuze, Unify, and several more.

The partnership will focus on video collaboration to deliver integrated audio/video solutions to the platforms of EPOS’ current strategic partners such as Microsoft.

They are pushing at an open door. The video conferencing market is predicted to grow from an estimated $1.8bn to more than $2.8bn by 2022, according to some studies.

Meet the robots Toyota is bringing to the 2020 Tokyo Olympic Games

Tokyo’s 2020 Summer Olympic and Paralympic Games are fast-approaching, and Toyota is playing a key role in on-site mobility and transportation. The Japanese automaker has unveiled five robots it’s also going to be bringing to the games, which will each help in some way to support athletes and attendees at avenues get around, get information, experience the games remotely, ferry food, drinks an equipment and much more. The robots range from humanoid to strictly purpose-built and functional in design.

[gallery ids="1858367,1858373"]

First up are two robot designs based on Tokyo’s official Olympic mascots, Miraitowa and Someity. These blue and pink big-eyed bots will be on-site at official venues acting as greeters and photo-ops, but they’re equipped with cameras and digital eyes that can offer expressions in response to human interaction. They’ll also be able to move their arms and legs, and part of the plan for deploying them is to potentially distribute them across Japan to offer kids in other cities a chance to get a taste of the games from afar.

T HR3 Humanoid Robot Latest compressor

T-HR3 offers a similar set of features, albeit in a very different design. This humanoid robot is a lot less ‘cute’ than the mascot bots, but has a lot more potential in terms of articulation. It’s also intended to provide a remote experience of what it’s like to be at the games, and can reproduce the movements of its mascot robot counterparts in real-time. T-HR3 can also stream images and sounds from the remote locations back to the Olympic site, acting as telepresence bots for Olympic fans off-site and mirroring their movements – they can “converse with and high-five athletes and others,” Toyota says specifically.

T TR1 Remote location communication Robot

Next up is T-TR1, a more traditional kind of telepresence robot that has a wheeled base, cameras ad a super large vertical display. It can show people at basically life-size scale, and let remote people chat in real-time with Olympic athletes and fans on-site to really feel like they’re there.

HSR Human Support Robot DSR Delivery Support Robot

This next robot is actually a pair – Human Support Robot (HSR) and Delivery Support Robot (DSR). The HSR is basically a robotic usher, providing guidance to seats for guests at venues, and also transporting some snacks, souvenirs and other light cargo to them at their seat while they watch the games. DSR, which is all-new for the Olympics, is more dedicated to delivering drinks and concessions to attendees on-site at venues, with people being able to order from a dedicated tablet – a modern replacement for hawkers walking the stands with trays of popcorn, peanuts and drinks.

FSR Field Support Robot Field Event Support Robot

Last but not least is Field Support Robot (FSR). This box-on-wheels should actually play an instrumental role, specifically supporting those game that involve hurling something as far as you can throw it. FSR’s entire purpose is to take the best route possible to retrieve things like javelins and shot-puts and return them to where they’re needed once thrown and recorded. They live to fetch.

Sadly, we’re not yet at the point where the robots are actually competing against the humans in a ‘winner-takes-all’ battle for the future of the planet. But these should nonetheless provide a look at what our future might be when robots are much more common in daily life, supporting humans in similar ways when there isn’t an international competition of the best-performing athletes in the world going on.

Location-based virtual reality is increasing its footprint in the U.S.

Earlier this year, in a small, grey-walled storefront inside a very large mall in Torrance, Calif. (just past the AMC Center) , the virtual reality game-maker Survios planted its first flag in the market for location-based gaming.

It’s one of several companies (many based in Los Angeles) that are turning the city into a hub for anyone looking to experience the thrill of immersive gaming.

While Survios’ offering is more akin to the virtual arcades cropping up in cities across the country and around the world (including Dubai, New York, Seoul, and Tokyo), other companies like the Los Angeles-based Two Bit Circus and Lindon, Utah’s The Void are creating site specific game experiences that promise a different kind of approach to virtual reality.

For Survios and other companies that have placed multi-million dollar bets on the viability of virtual reality, the move to location-based gaming isn’t a matter of choice. It’s a matter of survival thanks to the persistent lack of demand from consumers. 

Sales of head-mounted displays began to climb out of their doldrums late last year, and are expected to surpass 1.5 million head mounted displays sold in 2018, according to data from Canalys. But that’s still a far smaller market than the 10 million game consoles that were sold in the U.S. alone in 2017 (not to mention the roughly 32 million consoles sold at the market’s peak in 2008), according data on the Statista website

The benefits of location-based experiences are clear. The cost of premium headsets and gaming systems prohibit most U.S. households from getting the gear in their hands and until those costs come down, out-of-home experiences provide the best way to get consumers comfortable with the technology.

That’s been the tactic ever since Nolan Bushnell and Ted Dabney launched Computer Space in 1971 with the first coin-operated computer game for arcades.

And one that VRWorld brought (with much fanfare) to virtual reality in the U.S. with the debut of its three-floor gaming hub near the Empire State Building in the heart of New York.

That experience, a more extravagant investment than Survios’ humble multi-bay storefront, was one of the first in the U.S. to commit to the sensory overload that is virtual reality. By 2018, New York was home to at least seven virtual reality spaces where users could experience the technology, according to The New York Times.

And while it’s hard to recreate a truly immersive, mobile game experience in the home, the ability to access cinematic quality production values, a physical space purpose-built for immersive game play, and the intellectual property of some of Hollywood’s most enduring brands (like The Void’s Star Wars experience) can make for a compelling pitch to consumers.

That’s the hope of people like Nancy Bennett, an entertainment industry veteran who was brought on as the Chief Creative Officer at Two Bit Circus.

“What’s cool about VR and a differentiator of the medium is that it gives you embodiment,” Bennett says. “There’s no other medium that does that.”

Bennett knows a thing or two about entertainment. A producer with MTV Networks, the founder of the collaborative game development platform Squarepushers Inc. and a celebrated creator of virtual reality projects for the National Football League, the National Basketball Association, Bennett won the Lumiere award for best music VR experience for her work on the “One At a Time” video for Alex Aiono. 

From haptic platforms and motion floors that simulate the ability to walk around a space, the location based experience will offer a more fully immersive platform that can lend itself to more interesting narratives, says Bennett.

For Bennett, the vision of a place like Two Bit Circus, or the experiences on offer from other location based platforms are about the combination of narrative and technology in a way that can provide verisimilitude to someone strapped into a headset.

She, and others in the location-based community, look to immersive theater like Sleep No More as a model for how to proceed. “Immersive theater is absolutely the platform that will help drag us along,” Bennett says. 

At Two Bit Circus, which raised $15 million from investors last January, virtual reality will be about 20% of the experiences on offer. The company’s inaugural space in Los Angeles will also avail itself of projection mapping, augmented reality and other ways to immerse and entertain, Bennett promises.

But immersion will be at the heart of it all, she said. “Those kinds of mixed immersive experiences are going to be de rigueur,” according to Bennett. “And locations are going to be the only places where you can pull that off.”

Bennett sees the industry offering different tiers of immersive entertainment. With virtual reality arcades like Survios’ in Torrance operating on one level and more highly immersive experiences like The Void and Baobab Studios operating on another.

It’s one reason why companies like Cinemark have announced that they’re working with The Void and other immersive, location-based virtual reality companies to create experiences in their theaters.

“Really it’s about what serves the creative goal,” says Bennett. “What I think is really cool is the opportunity to mash up the fast prototyping of the community into one space to get people to play. It isn’t just VR. There’s also new forms of play and arcades that are possible and interactive audience participation for content creation.”

Even with the wow-factor of the experience, it may not be enough to buck industry trends. IMAX was one of the first companies to carve out immersive virtual reality spaces in its theaters, but given its woeful performance in the first quarter of 2018, those efforts are now on hold, according to it chief executive Richard Gelfond.

“At this time, we do not anticipate opening additional VR centers, or making a meaningful future investments in the initiative,” he told analysts during the company’s first quarter earnings call.

It’s a dramatic change for a company that was touting its entrance into the location based market just a year earlier.

IMAX’s stumble belies the international success of location-based gaming. In this, Asia leads the way with virtual reality outposts like the Viveland theme park in China. An existing infrastructure of internet cafes meant that Asian gaming hubs could just throw virtual reality hardware into their mix of offerings and continue to attract an audience.

Meanwhile, companies in the U.S. need to depend on purpose built spaces for virtual reality gaming thanks to the dominance of in-home gaming consoles (which overtook arcade gaming at least a decade ago). The lack of similar out-of-home spaces led to IMAX deciding to set up their own experiences — and other movie theaters and amusement parks following suit.

And there’s still the chance that in-home virtual reality will be able to pick up the pace and boost adoption more quickly than the market expects.

Analysts for the industry tracker Canalys forecast that the industry will sell nearly 10 million units in 2021, on par with the (shrinking) console market. Standalone virtual reality headsets are expected to push the market to 7.6 million units sold by the end of 2018, according to Canalys.

Still, for the immediate future, for those looking to get the full benefit of a virtual reality experience, their best bet is to find the nearest Void experience and battle some storm troopers, check out an arcade, or wait for the unveiling of Two Bit Circus’ first facility later this year.