Longevity startup Gero AI has a mobile API for quantifying health changes

Sensor data from smartphones and wearables can meaningfully predict an individual’s ‘biological age’ and resilience to stress, according to Gero AI.

The ‘longevity’ startup — which condenses its mission to the pithy goal of “hacking complex diseases and aging with Gero AI” — has developed an AI model to predict morbidity risk using ‘digital biomarkers’ that are based on identifying patterns in step-counter sensor data which tracks mobile users’ physical activity.

A simple measure of ‘steps’ isn’t nuanced enough on its own to predict individual health, is the contention. Gero’s AI has been trained on large amounts of biological data to spots patterns that can be linked to morbidity risk. It also measures how quickly a personal recovers from a biological stress — another biomarker that’s been linked to lifespan; i.e. the faster the body recovers from stress, the better the individual’s overall health prognosis.

A research paper Gero has had published in the peer-reviewed biomedical journal Aging explains how it trained deep neural networks to predict morbidity risk from mobile device sensor data — and was able to demonstrate that its biological age acceleration model was comparable to models based on blood test results.

Another paper, due to be published in the journal Nature Communications later this month, will go into detail on its device-derived measurement of biological resilience.

The Singapore-based startup, which has research roots in Russia — founded back in 2015 by a Russian scientist with a background in theoretical physics — has raised a total of $5 million in seed funding to date (in two tranches).

Backers come from both the biotech and the AI fields, per co-founder Peter Fedichev. Its investors include Belarus-based AI-focused early stage fund, Bulba Ventures (Yury Melnichek). On the pharma side, it has backing from some (unnamed) private individuals with links to Russian drug development firm, Valenta. (The pharma company itself is not an investor).

Fedichev is a theoretical physicist by training who, after his PhD and some ten years in academia, moved into biotech to work on molecular modelling and machine learning for drug discovery — where he got interested in the problem of ageing and decided to start the company.

As well as conducting its own biological research into longevity (studying mice and nematodes), it’s focused on developing an AI model for predicting the biological age and resilience to stress of humans — via sensor data captured by mobile devices.

“Health of course is much more than one number,” emphasizes Fedichev. “We should not have illusions about that. But if you are going to condense human health to one number then, for a lot of people, the biological age is the best number. It tells you — essentially — how toxic is your lifestyle… The more biological age you have relative to your chronological age years — that’s called biological acceleration — the more are your chances to get chronic disease, to get seasonal infectious diseases or also develop complications from those seasonal diseases.”

Gero has recently launched a (paid, for now) API, called GeroSense, that’s aimed at health and fitness apps so they can tap up its AI modelling to offer their users an individual assessment of biological age and resilience (aka recovery rate from stress back to that individual’s baseline).

Early partners are other longevity-focused companies, AgelessRx and Humanity Inc. But the idea is to get the model widely embedded into fitness apps where it will be able to send a steady stream of longitudinal activity data back to Gero, to further feed its AI’s predictive capabilities and support the wider research mission — where it hopes to progress anti-ageing drug discovery, working in partnerships with pharmaceutical companies.

The carrot for the fitness providers to embed the API is to offer their users a fun and potentially valuable feature: A personalized health measurement so they can track positive (or negative) biological changes — helping them quantify the value of whatever fitness service they’re using.

“Every health and wellness provider — maybe even a gym — can put into their app for example… and this thing can rank all their classes in the gym, all their systems in the gym, for their value for different kinds of users,” explains Fedichev.

“We developed these capabilities because we need to understand how ageing works in humans, not in mice. Once we developed it we’re using it in our sophisticated genetic research in order to find genes — we are testing them in the laboratory — but, this technology, the measurement of ageing from continuous signals like wearable devices, is a good trick on its own. So that’s why we announced this GeroSense project,” he goes on.

“Ageing is this gradual decline of your functional abilities which is bad but you can go to the gym and potentially improve them. But the problem is you’re losing this resilience. Which means that when you’re [biologically] stressed you cannot get back to the norm as quickly as possible. So we report this resilience. So when people start losing this resilience it means that they’re not robust anymore and the same level of stress as in their 20s would get them [knocked off] the rails.

“We believe this loss of resilience is one of the key ageing phenotypes because it tells you that you’re vulnerable for future diseases even before those diseases set in.”

“In-house everything is ageing. We are totally committed to ageing: Measurement and intervention,” adds Fedichev. “We want to building something like an operating system for longevity and wellness.”

Gero is also generating some revenue from two pilots with “top range” insurance companies — which Fedichev says it’s essentially running as a proof of business model at this stage. He also mentions an early pilot with Pepsi Co.

He sketches a link between how it hopes to work with insurance companies in the area of health outcomes with how Elon Musk is offering insurance products to owners of its sensor-laden Teslas, based on what it knows about how they drive — because both are putting sensor data in the driving seat, if you’ll pardon the pun. (“Essentially we are trying to do to humans what Elon Musk is trying to do to cars,” is how he puts it.)

But the nearer term plan is to raise more funding — and potentially switch to offering the API for free to really scale up the data capture potential.

Zooming out for a little context, it’s been almost a decade since Google-backed Calico launched with the moonshot mission of ‘fixing death’. Since then a small but growing field of ‘longevity’ startups has sprung up, conducting research into extending (in the first instance) human lifespan. (Ending death is, clearly, the moonshot atop the moonshot.) 

Death is still with us, of course, but the business of identifying possible drugs and therapeutics to stave off the grim reaper’s knock continues picking up pace — attracting a growing volume of investor dollars.

The trend is being fuelled by health and biological data becoming ever more plentiful and accessible, thanks to open research data initiatives and the proliferation of digital devices and services for tracking health, set alongside promising developments in the fast-evolving field of machine learning in areas like predictive healthcare and drug discovery.

Longevity has also seen a bit of an upsurge in interest in recent times as the coronavirus pandemic has concentrated minds on health and wellness, generally — and, well, mortality specifically.

Nonetheless, it remains a complex, multi-disciplinary business. Some of these biotech moonshots are focused on bioengineering and gene-editing — pushing for disease diagnosis and/or drug discovery.

Plenty are also — like Gero —  trying to use AI and big data analysis to better understand and counteract biological ageing, bringing together experts in physics, maths and biological science to hunt for biomarkers to further research aimed at combating age-related disease and deterioration.

Another recent example is AI startup Deep Longevity, which came out of stealth last summer — as a spinout from AI drug discovery startup Insilico Medicine — touting an AI ‘longevity as a service’ system which it claims can predict an individual’s biological age “significantly more accurately than conventional methods” (and which it also hopes will help scientists to unpick which “biological culprits drive aging-related diseases”, as it put it).

Gero AI is taking a different tack toward the same overarching goal — by honing in on data generated by activity sensors embedded into the everyday mobile devices people carry with them (or wear) as a proxy signal for studying their biology.

The advantage being that it doesn’t require a person to undergo regular (invasive) blood tests to get an ongoing measure of their own health. Instead our personal device can generate proxy signals for biological study passively — at vast scale and low cost. So the promise of Gero’s ‘digital biomarkers’ is they could democratize access to individual health prediction.

And while billionaires like Peter Thiel can afford to shell out for bespoke medical monitoring and interventions to try to stay one step ahead of death, such high end services simply won’t scale to the rest of us.

If its digital biomarkers live up to Gero’s claims, its approach could, at the least, help steer millions towards healthier lifestyles, while also generating rich data for longevity R&D — and to support the development of drugs that could extend human lifespan (albeit what such life-extending pills might cost is a whole other matter).

The insurance industry is naturally interested — with the potential for such tools to be used to nudge individuals towards healthier lifestyles and thereby reduce payout costs.

For individuals who are motivated to improve their health themselves, Fedichev says the issue now is it’s extremely hard for people to know exactly which lifestyle changes or interventions are best suited to their particular biology.

For example fasting has been shown in some studies to help combat biological ageing. But he notes that the approach may not be effective for everyone. The same may be true of other activities that are accepted to be generally beneficial for health (like exercise or eating or avoiding certain foods).

Again those rules of thumb may have a lot of nuance, depending on an individual’s particular biology. And scientific research is, inevitably, limited by access to funding. (Research can thus tend to focus on certain groups to the exclusion of others — e.g. men rather than women; or the young rather than middle aged.)

This is why Fedichev believes there’s a lot of value in creating a measure than can address health-related knowledge gaps at essentially no individual cost.

Gero has used longitudinal data from the UK’s biobank, one of its research partners, to verify its model’s measurements of biological age and resilience. But of course it hopes to go further — as it ingests more data. 

“Technically it’s not properly different what we are doing — it just happens that we can do it now because there are such efforts like UK biobank. Government money and also some industry sponsors money, maybe for the first time in the history of humanity, we have this situation where we have electronic medical records, genetics, wearable devices from hundreds of thousands of people, so it just became possible. It’s the convergence of several developments — technological but also what I would call ‘social technologies’ [like the UK biobank],” he tells TechCrunch.

“Imagine that for every diet, for every training routine, meditation… in order to make sure that we can actually optimize lifestyles — understand which things work, which do not [for each person] or maybe some experimental drugs which are already proved [to] extend lifespan in animals are working, maybe we can do something different.”

“When we will have 1M tracks [half a year’s worth of data on 1M individuals] we will combine that with genetics and solve ageing,” he adds, with entrepreneurial flourish. “The ambitious version of this plan is we’ll get this million tracks by the end of the year.”

Fitness and health apps are an obvious target partner for data-loving longevity researchers — but you can imagine it’ll be a mutual attraction. One side can bring the users, the other a halo of credibility comprised of deep tech and hard science.

“We expect that these [apps] will get lots of people and we will be able to analyze those people for them as a fun feature first, for their users. But in the background we will build the best model of human ageing,” Fedichev continues, predicting that scoring the effect of different fitness and wellness treatments will be “the next frontier” for wellness and health (Or, more pithily: “Wellness and health has to become digital and quantitive.”)

“What we are doing is we are bringing physicists into the analysis of human data. Since recently we have lots of biobanks, we have lots of signals — including from available devices which produce something like a few years’ long windows on the human ageing process. So it’s a dynamical system — like weather prediction or financial market predictions,” he also tells us.

“We cannot own the treatments because we cannot patent them but maybe we can own the personalization — the AI that personalized those treatments for you.”

From a startup perspective, one thing looks crystal clear: Personalization is here for the long haul.


Cognixion’s brain-monitoring headset enables fluid communication for people with severe disabilities

Of the many frustrations of having a severe motor impairment, the difficulty of communicating must surely be among the worst. The tech world has not offered much succor to those affected by things like locked-in syndrome, ALS, and severe strokes, but startup Cognixion aims to with a novel form of brain monitoring that, combined with a modern interface, could make speaking and interaction far simpler and faster.

The company’s One headset tracks brain activity closely in such a way that the wearer can direct a cursor — reflected on a visor like a heads-up display — in multiple directions or select from various menus and options. No physical movement is needed, and with the help of modern voice interfaces like Alexa, the user can not only communicate efficiently but freely access all kinds of information and content most people take for granted.

But it’s not a miracle machine, and it isn’t a silver bullet. Here’s where how it got started.

Overhauling decades-old brain tech

Everyone with a motor impairment has different needs and capabilities, and there are a variety of assistive technologies that cater to many of these needs. But many of these techs and interfaces are years or decades old — medical equipment that hasn’t been updated for an era of smartphones and high-speed mobile connections.

Some of the most dated interfaces, unfortunately, are those used by people with the most serious limitations: those whose movements are limited to their heads, faces, eyes — or even a single eyelid, like Jean-Dominique Bauby, the famous author of “The Diving Bell and the Butterfly.”

One of the tools in the toolbox is the electroencephalogram, or EEG, which involves detecting activity in the brain via patches on the scalp that record electrical signals. But while they’re useful in medicine and research in many ways, EEGs are noisy and imprecise — more for finding which areas of the brain are active than, say, which sub-region of the sensory cortex or the like. And of course you have to wear a shower cap wired with electrodes (often greasy with conductive gel) — it’s not the kind of thing anyone wants to do for more than an hour, let alone all day every day.

Yet even among those with the most profound physical disabilities, cognition is often unimpaired — as indeed EEG studies have helped demonstrate. It made Andreas Forsland, co-founder and CEO of Cognixion, curious about further possibilities for the venerable technology: “Could a brain-computer interface using EEG be a viable communication system?”

He first used EEG for assistive purposes in a research study some five years ago. They were looking into alternative methods of letting a person control an on-screen cursor, among them an accelerometer for detecting head movements, and tried integrating EEG readings as another signal. But it was far from a breakthrough.

A modern lab with an EEG cap wired to a receiver and laptop – this is an example of how EEG is commonly used.

He ran down the difficulties: “With a read-only system, the way EEG is used today is no good; other headsets have slow sample rates and they’re not accurate enough for a real-time interface. The best BCIs are in a lab, connected to wet electrodes — it’s messy, it’s really a non-starter. So how do we replicate that with dry, passive electrodes? We’re trying to solve some very hard engineering problems here.”

The limitations, Forsland and his colleagues found, were not so much with the EEG itself as with the way it was carried out. This type of brain monitoring is meant for diagnosis and study, not real-time feedback. It would be like taking a tractor to a drag race. Not only do EEGs often work with a slow, thorough check of multiple regions of the brain that may last several seconds, but the signal it produces is analyzed by dated statistical methods. So Cognixion started by questioning both practices.

Improving the speed of the scan is more complicated than overclocking the sensors or something. Activity in the brain must be inferred by collecting a certain amount of data. But that data is collected passively, so Forsland tried bringing an active element into it: a rhythmic electric stimulation that is in a way reflected by the brain region, but changed slightly depending on its state — almost like echolocation.

The Cognixion One headset with its dry EEG terminals visible.

They detect these signals with a custom set of six EEG channels in the visual cortex area (up and around the back of your head), and use a machine learning model to interpret the incoming data. Running a convolutional neural network locally on an iPhone — something that wasn’t really possible a couple years ago — the system can not only tease out a signal in short order but make accurate predictions, making for faster and smoother interactions.

The result is sub-second latency with 95-100 percent accuracy in a wireless headset powered by a mobile phone. “The speed, accuracy and reliability are getting to commercial levels —  we can match the best in class of the current paradigm of EEGs,” said Forsland.

Dr. William Goldie, a clinical neurologist who has used and studied EEGs and other brain monitoring techniques for decades (and who has been voluntarily helping Cognixion develop and test the headset), offered a positive evaluation of the technology.

“There’s absolutely evidence that brainwave activity responds to thinking patterns in predictable ways,” he noted. This type of stimulation and response was studied years ago. “It was fascinating, but back then it was sort of in the mystery magic world. Now it’s resurfacing with these special techniques and the computerization we have these days. To me it’s an area that’s opening up in a manner that I think clinically could be dramatically effective.”

BCI, meet UI

The first thing Forsland told me was “We’re a UI company.” And indeed even such a step forward in neural interfaces as he later described means little if it can’t be applied to the problem at hand: helping people with severe motor impairment to express themselves quickly and easily.

Sad to say, it’s not hard to imagine improving on the “competition,” things like puff-and-blow tubes and switches that let users laboriously move a cursor right, right a little more, up, up a little more, then click: a letter! Gaze detection is of course a big improvement over this, but it’s not always an option (eyes don’t always work as well as one would like) and the best eye-tracking solutions (like a Tobii Dynavox tablet) aren’t portable.

Why shouldn’t these interfaces be as modern and fluid as any other? The team set about making a UI with this and the capabilities of their next-generation EEG in mind.

Image of the target Cognixion interface as it might appear to a user, with buttons for yes, no, phrases and tools.

Image Credits: Cognixion

Their solution takes bits from the old paradigm and combines them with modern virtual assistants and a radial design that prioritizes quick responses and common needs. It all runs in an app on an iPhone, the display of which is reflected in a visor, acting as a HUD and outward-facing display.

In easy reach of, not to say a single thought but at least a moment’s concentration or a tilt of the head, are everyday questions and responses — yes, no, thank you, etc. Then there are slots to put prepared speech into — names, menu orders, and so on. And then there’s a keyboard with word- and sentence-level prediction that allows common words to be popped in without spelling them out.

“We’ve tested the system with people who rely on switches, who might take 30 minutes to make 2 selections. We put the headset on a person with cerebral palsy, and she typed our her name and hit play in 2 minutes,” Forsland said. “It was ridiculous, everyone was crying.”

Goldie noted that there’s something of a learning curve. “When I put it on, I found that it would recognize patterns and follow through on them, but it also sort of taught patterns to me. You’re training the system, and it’s training you — it’s a feedback loop.”

“I can be the loudest person in the room”

One person who has found it extremely useful is Chris Benedict, a DJ, public speaker, and disability advocate who himself has Dyskinetic Cerebral Palsy. It limits his movements and ability to speak, but doesn’t stop him from spinning (digital) records at various engagements, however, or from explaining his experience with Cognixion’s One headset over email. (And you can see him demonstrating it in person in the video above.)

DJ Chris Benedict wears the Cognixion Headset in a bright room.

Image Credits: Cognixion

“Even though it’s not a tool that I’d need all the time it’s definitely helpful in aiding my communication,” he told me. “Especially when I need to respond quickly or am somewhere that is noisy, which happens often when you are a DJ. If I wear it with a Bluetooth speaker I can be the loudest person in the room.” (He always has a speaker on hand, since “you never know when you might need some music.”)

The benefits offered by the headset give some idea of what is lacking from existing assistive technology (and what many people take for granted).

“I can use it to communicate, but at the same time I can make eye contact with the person I’m talking to, because of the visor. I don’t have to stare at a screen between me and someone else. This really helps me connect with people,” Benedict explained.

“Because it’s a headset I don’t have to worry about getting in and out of places, there is no extra bulk added to my chair that I have to worry about getting damaged in a doorway. The headset is balanced too, so it doesn’t make my head lean back or forward or weigh my neck down,” he continued. “When I set it up to use the first time it had me calibrate, and it measured my personal range of motion so the keyboard and choices fit on the screen specifically for me. It can also be recalibrated at any time, which is important because not every day is my range of motion the same.”

Alexa, which has been extremely helpful to people with a variety of disabilities due to its low cost and wide range of compatible devices, is also part of the Cognixion interface, something Benedict appreciates, having himself adopted the system for smart home and other purposes. “With other systems this isn’t something you can do, or if it is an option, it’s really complicated,” he said.

Next steps

As Benedict demonstrates, there are people for whom a device like Cognixion’s makes a lot of sense, and the hope is it will be embraced as part of the necessarily diverse ecosystem of assistive technology.

Forsland said that the company is working closely with the community, from users to clinical advisors like Goldie and other specialists, like speech therapists, to make the One headset as good as it can be. But the hurdle, as with so many devices in this class, is how to actually put it on people’s heads — financially and logistically speaking.

Cognixion is applying for FDA clearance to get the cost of the headset — which, being powered by a phone, is not as high as it would be with an integrated screen and processor — covered by insurance. But in the meantime the company is working with clinical and corporate labs that are doing neurological and psychological research. Places where you might find an ordinary, cumbersome EEG setup, in other words.

The company has raised funding and is looking for more (hardware development and medical pursuits don’t come cheap), and has also collected a number of grants.

The One headset may still be some years away from wider use (the FDA is never in a hurry), but that allows the company time to refine the device and include new advances. Unlike many other assistive devices, for example a switch or joystick, this one is largely software-limited, meaning better algorithms and UI work will significantly improve it. While many wait for companies like Neuralink to create a brain-computer interface for the modern era, Cognixion has already done so for a group of people who have much more to gain from it.

You can learn more about the Cognixion One headset and sign up to receive the latest at its site here.

Computer vision inches towards ‘common sense’ with Facebook’s latest research

Machine learning is capable of doing all sorts of things as long as you have the data to teach it how. That’s not always easy, and researchers are always looking for a way to add a bit of “common sense” to AI so you don’t have to show it 500 pictures of a cat before it gets it. Facebook’s newest research takes a big step towards reducing the data bottleneck.

The company’s formidable AI research division has been working on how to advance and scale things like advanced computer vision algorithms for years now, and has made steady progress, generally shared with the rest of the research community. One interesting development Facebook has pursued in particular is what’s called “semi-supervised learning.”

Generally when you think of training an AI, you think of something like the aforementioned 500 pictures of cats — images that have been selected and labeled (which can mean outlining the cat, putting a box around the cat, or just saying there’s a cat in there somewhere) so that the machine learning system can put together an algorithm to automate the process of cat recognition. Naturally if you want to do dogs or horses, you need 500 dog pictures, 500 horse pictures, etc — it scales linearly, which is a word you never want to see in tech.

Semi-supervised learning, related to “unsupervised” learning, involves figuring out important parts of a dataset without any labeled data at all. It doesn’t just go wild, there’s still structure; for instance, imagine you give the system a thousand sentences to study, then showed it ten more that have several of the words missing. The system could probably do a decent job filling in the blanks just based on what it’s seen in the previous thousand. But that’s not so easy to do with images and video — they aren’t as straightforward or predictable.

But Facebook researchers have shown that while it may not be easy, it’s possible and in fact very effective. The DINO system (which stands rather unconvincingly for “DIstillation of knowledge with NO labels”) is capable of learning to find objects of interest in videos of people, animals, and objects quite well without any labeled data whatsoever.

Animation showing four videos and the AI interpretation of the objects in them.

Image Credits: Facebook

It does this by considering the video not as a sequence of images to be analyzed one by one in order, but as an complex, interrelated set,like the difference between “a series of words” and “a sentence.” By attending to the middle and the end of the video as well as the beginning, the agent can get a sense of things like “an object with this general shape goes from left to right.” That information feeds into other knowledge, like when an object on the right overlaps with the first one, the system knows they’re not the same thing, just touching in those frames. And that knowledge in turn can be applied to other situations. In other words, it develops a basic sense of visual meaning, and does so with remarkably little training on new objects.

This results in a computer vision system that’s not only effective — it performs well compared with traditionally trained systems — but more relatable and explainable. For instance, while an AI that has been trained with 500 dog pictures and 500 cat pictures will recognize both, it won’t really have any idea that they’re similar in any way. But DINO — although it couldn’t be specific — gets that they’re similar visually to one another, more so anyway than they are to cars, and that metadata and context is visible in its memory. Dogs and cats are “closer” in its sort of digital cognitive space than dogs and mountains. You can see those concepts as little blobs here — see how those of a type stick together:

Animated diagram showing how concepts in the machine learning model stay close together.

Image Credits: Facebook

This has its own benefits, of a technical sort we won’t get into here. If you’re curious, there’s more detail in the papers linked in Facebook’s blog post.

There’s also an adjacent research project, a training method called PAWS, which further reduces the need for labeled data. PAWS combines some of the ideas of semi-supervised learning with the more traditional supervised method, essentially giving the training a boost by letting it learn from both the labeled and unlabeled data.

Facebook of course needs good and fast image analysis for its many user-facing (and secret) image-related products, but these general advances to the computer vision world will no doubt be welcomed by the developer community for other purposes.

Erase All Kittens raises $1M Seed round for Mario-style game which teaches girls to code

Erase All Kittens (EAK) is an EdTech startup that created a ‘Mario-style’ web-based game designed for kids aged 8-12. However, the game has a twist: it places an emphasis on inspiring girls to code (since let’s face it, most coding tools are created by men). After reaching 160,000 players in over 100 countries, it’s now raised a $1M Seed funding led by Twinkl Educational Publishing, with participation from first investor Christian Reyntjens of the A Black Square family office, alongside angel investors, including one of the founders of Shazam.

While the existing EAK game is free, a new game launched in July will be paid for, further boosting the product’s business model.

EAK says its research shows that some 55% of its players are girls, and 95% want to learn more about coding after playing its game. EAK is currently being used in over 3,000 schools, mostly in the UK and US, and its traction increased by 500% during the lockdowns associated with the pandemic.

It’s Erase All Kittens’ contention that coding education tools for children have been largely built by men and so naturally appeal more to boys. With most teaching repetitive coding, in a very rigid, instructional way, it tends to appeal more to boys than girls, says EAK.

The female-founded team has a platform for changing the perception that kids, especially girls, have of coding. After R&D of two years, it came up with a game designed to teach kids and girls as young as 8 skills such as HTML, CSS, and Javascript through highly gamified, story-driven gameplay. Kids get to chat with characters on their journey, for example, a serial entrepreneur unicorn mermaid called Tarquin Glitterquiff.

“Players edit the code that governs the game environment, building and fixing levels as they play in order to save kittens in a fantasy internet universe,” said Dee Saigal, cofounder, CEO and creative director. Saigal is joined by co-founder Leonie Van Der Linde; CTO Rex Van Der Spuy; Senior Games Developer Jeremy Keen; and 2D Games Artist Mikhail Malkin.

Erase All Kittens game

Erase All Kittens game

The existing game teaches HTML skills and how to create URLs, and the new game (released in July this year) will teach HTML, CSS, and Javascript skills – bridging the huge gap between kids learning the concepts and being able to create on the web like developers.

Said Saigal: “We’re designing a coding game that girls genuinely love – one that places a huge emphasis on creativity. Girls can see instant results as they code, there are different ways to progress through the game, and learning is seamlessly blended with storytelling.”

Saigal said: “When I was younger I wanted to be a games designer. I loved coming up with ideas for games but coding had always seemed like an impossible task. We weren’t taught coding at school, and I couldn’t see anyone who looked like me making games, so I didn’t think it was something I could do.”

“Whilst researching our target audience, we found that one of the biggest obstacles for girls still begins with gender stereotypes from an early age. By the time girls reach school, this snowballs into a lack of confidence in STEM skills and lower expectations from teachers, which in turn can lead to lower performance—a gap that only widens as girls get older.”

EAK’s competitors include Code Kingdoms, Swift Playgrounds and CodeCombat. But Saigal says these games tend to appeal far more to boys than to girls.

The new game (see below) will be sold to schools and parents, globally. EAK will also be carrying out a one-for-one scheme, where for every school account purchased, one will be donated to underserved schools via partnerships with tech companies, educational organizations, and NGOs.

Jonathan Seaton, Co-founder and CEO at Twinkl and Director of TwinklHive, said: “We’re really excited to partner with Erase All Kittens, as a digital company Twinkl recognizes the importance of preparing children to succeed in the digital age and we believe through this partnership we can really make a difference.”

“The team is particularly excited about helping further Erase All Kitten’s mission to empower girls and give them the same opportunities to learn to code and build their own digital creations. Ensuring that all children have equal access to opportunities to learn is at the heart of Twinkl’s vision and a key motivation in the development of this partnership for both organizations.”

Erase All Kittens

Erase All Kittens

Erase All Kittens says it is addressing the global skills gap, where the gender gap is increasingly widening. According to PWC, just 24% of the tech workforce is female and women make up just 12% of all engineers, while only 3% of female students in the UK list tech as their first career choice.

Research by Childwise found that 90% of girls give up on coding after first trying it, and if they lose interest in STEM subject by the age of 11, they never recover from that. This is a huge and growing problem for the tech industry and for investors.

Materials Zone raises $6M for its materials discovery platform

Materials Zone, a Tel Aviv-based startup that uses AI to speed up materials research, today announced that it has raised a $6 million seed funding round led by Insight Partners, with participation from crowdfunding platform OurCrowd.

The company’s platform consists of a number of different tools, but at the core is a database that takes in data from scientific instruments, manufacturing facilities, lab equipment, external databases, published articles, Excel sheets and more, and then parses it and standardizes it. Simply having this database, the company argues, is a boon for researchers, who can then also visualize it as needed.

Image Credits: Materials Zone

“In order to develop new technologies and physical products, companies must first understand the materials that comprise those products, as well as those materials’ properties,” said Materials Zone founder and CEO Dr. Assaf Anderson. “Understanding the science of materials has therefore become a driving force behind innovation. However, the data behind materials R&D and production has traditionally been poorly managed, unstructured, and underutilized, often leading to redundant experiments, limited capacity to build on past experience, and an inability to effectively collaborate, which inevitably wastes countless dollars and man-hours.”

Image Credits: Materials Zone

Before founding Materials Zone, Anderson spent time at the Bar Ilan University’s Institute for Nanotechnology and Advanced Materials, where he was the head of the Combinatorial Materials lab.

Assaf Anderson, Ph.D., founder and CEO of Materials Zone

Assaf Anderson, PhD, founder/CEO of Materials Zone. Image Credits: Materials Zone

“As a materials scientist, I have experienced R&D challenges firsthand, thereby gaining an understanding of how R&D can be improved,” Anderson said. “We developed our platform with our years of experience in mind, leveraging innovative AI/ML technologies to create a unique solution for these problems.”

He noted that in order to, for example, develop a new photovoltaic transparent window, it would take thousands of experiments to find the right core materials and their parameters. The promise of Materials Zone is that it can make this process faster and cheaper by aggregating and standardizing all of this data and then offer data and workflow management tools to work with it. Meanwhile, the company’s analytical and machine learning tools can help researchers interpret this data.


SLAIT’s real-time sign language translation promises more accessible online communication

Sign language is used by millions of people around the world, but unlike Spanish, Mandarin or even Latin, there’s no automatic translation available for those who can’t use it. SLAIT claims the first such tool available for general use, which can translate around 200 words and simple sentences to start — using nothing but an ordinary computer and webcam.

People with hearing impairments, or other conditions that make vocal speech difficult, number in the hundreds of millions, rely on the same common tech tools as the hearing population. But while emails and text chat are useful and of course very common now, they aren’t a replacement for face-to-face communication, and unfortunately there’s no easy way for signing to be turned into written or spoken words, so this remains a significant barrier.

We’ve seen attempts at automatic sign language (usually American/ASL) translation for years and years: in 2012 Microsoft awarded its Imagine Cup to a student team that tracked hand movements with gloves; in 2018 I wrote about SignAll, which has been working on a sign language translation booth using multiple cameras to give 3D positioning; and in 2019 I noted that a new hand-tracking algorithm called MediaPipe, from Google’s AI labs, could lead to advances in sign detection. Turns out that’s more or less exactly what happened.

SLAIT is a startup built out of research done at the Aachen University of Applied Sciences in Germany, where co-founder Antonio Domènech built a small ASL recognition engine using MediaPipe and custom neural networks. Having proved the basic notion, Domènech was joined by co-founders Evgeny Fomin and William Vicars to start the company; they then moved on to building a system that could recognize first 100, and now 200 individual ASL gestures and some simple sentences. The translation occurs offline, and in near real time on any relatively recent phone or computer.

Animation showing ASL signs being translated to text, and spoken words being transcribed to text back.

They plan to make it available for educational and development work, expanding their dataset so they can improve the model before attempting any more significant consumer applications.

Of course, the development of the current model was not at all simple, though it was achieved in remarkably little time by a small team. MediaPipe offered an effective, open-source method for tracking hand and finger positions, sure, but the crucial component for any strong machine learning model is data, in this case video data (since it would be interpreting video) of ASL in use — and there simply isn’t a lot of that available.

As they recently explained in a presentation for the DeafIT conference, the first team evaluated using an older Microsoft database, but found that a newer Australian academic database had more and better quality data, allowing for the creation of a model that is 92 percent accurate at identifying any of 200 signs in real time. They have augmented this with sign language videos from social media (with permission, of course) and government speeches that have sign language interpreters — but they still need more.

Animated image of a woman saying "deaf understand hearing" in ASL.

A GIF showing one of the prototypes in action — the consumer product won’t have a wireframe, obviously.Image Credits: Slait.ai

Their intention is to make the platform available to the deaf and ASL learner communities, who hopefully won’t mind their use of the system being turned to its improvement.

And naturally it could prove an invaluable tool in its present state, since the company’s translation model, even as a work in progress, is still potentially transformative for many people. With the amount of video calls going on these days and likely for the rest of eternity, accessibility is being left behind — only some platforms offer automatic captioning, transcription, summaries, and certainly none recognize sign language. But with SLAIT’s tool people could sign normally and participate in a video call naturally rather than using the neglected chat function.

“In the short term, we’ve proven that 200 word models are accessible and our results are getting better every day,” said SLAIT’s Evgeny Fomin. “In the medium term, we plan to release a consumer facing app to track sign language. However, there is a lot of work to do to reach a comprehensive library of all sign language gestures. We are committed to making this future state a reality. Our mission is to radically improve accessibility for the Deaf and hard of hearing communities.”

From left, Evgeny Fomin, Dominic Domènech, and Bill Vicars.Image Credits: Slait.ai

He cautioned that it will not be totally complete — just as translation and transcription in or to any language is only an approximation, the point is to provide practical results for millions of people, and a few hundred words goes a long way toward doing so. As data pours in, new words can be added to the vocabulary, and new multi-gesture phrases as well, and performance for the core set will improve.

Right now the company is seeking initial funding to get its prototype out and grow the team beyond the founding crew. Fomin said they have received some interest but want to make sure they connect with an investor who really understands the plan and vision.

When the engine itself has been built up to be more reliable by the addition of more data and the refining of the machine learning models, the team will look into further development and integration of the app with other products and services. For now the product is more of a proof of concept, but what a proof it is — with a bit more work SLAIT will have leapfrogged the industry and provided something that deaf and hearing people both have been wanting for decades.

Deepfake tech takes on satellite maps

While the concept of “deepfakes,” or AI-generated synthetic imagery, has been decried primarily in connection with involuntary depictions of people, the technology is dangerous (and interesting) in other ways as well. For instance, researchers have shown that it can be used to manipulate satellite imagery to produce real-looking — but totally fake — overhead maps of cities.

The study, led by Bo Zhao from the University of Washington, is not intended to alarm anyone but rather to show the risks and opportunities involved in applying this rather infamous technology to cartography. In fact their approach has as much in common with “style transfer” techniques — redrawing images in an impressionistic, crayon and arbitrary other fashions — than with deepfakes as they are commonly understood.

The team trained a machine learning system on satellite images of three different cities: Seattle, nearby Tacoma and Beijing. Each has its own distinctive look, just as a painter or medium does. For instance, Seattle tends to have larger overhanging greenery and narrower streets, while Beijing is more monochrome and — in the images used for the study — the taller buildings cast long, dark shadows. The system learned to associate details of a street map (like Google or Apple’s) with those of the satellite view.

The resulting machine learning agent, when given a street map, returns a realistic-looking faux satellite image of what that area would look like if it were in any of those cities. In the following image, the map corresponds to the top right satellite image of Tacoma, while the lower versions show how it might look in Seattle and Beijing.

Four images show a street map and a real satellite image of Tacoma, and two simulated satellite images of the same streets in Seattle and Beijing.

Image Credits: Zhao et al.

A close inspection will show that the fake maps aren’t as sharp as the real one, and there are probably some logical inconsistencies like streets that go nowhere and the like. But at a glance the Seattle and Beijing images are perfectly plausible.

One only has to think for a few minutes to conceive of uses for fake maps like this, both legitimate and otherwise. The researchers suggest that the technique could be used to simulate imagery of places for which no satellite imagery is available — like one of these cities in the days before such things were possible, or for a planned expansion or zoning change. The system doesn’t have to imitate another place altogether — it could be trained on a more densely populated part of the same city, or one with wider streets.

It could conceivably even be used, as this rather more whimsical project was, to make realistic-looking modern maps from ancient hand-drawn ones.

Should technology like this be bent to less constructive purposes, the paper also looks at ways to detect such simulated imagery using careful examination of colors and features.

The work challenges the general assumption of the “absolute reliability of satellite images or other geospatial data,” said Zhao in a UW news article, and certainly as with other media that kind thinking has to go by the wayside as new threats appear. You can read the full paper at the journal Cartography and Geographic Information Science.

This is your brain on Zoom

We all know these constant video calls are doing something to our brains. How else could we get tired and frazzled from sitting around in your own home all day? Well, now Microsoft has done a little brain science and found out that yeah, constant video calls do increase your stress and brain noise. Tell your boss!

The study had 14 people participate in eight half-hour video calls, divided into four a day — one day with ten-minute breaks between, and the other all in one block. The participants wore EEG caps: brain-monitoring gear that gives a general idea of types of activity in the old grey matter.

What they found is not particularly surprising, since we all have lived it for the last year (or more for already remote workers), but still important to show in testing. During the meeting block with no breaks, people showed higher levels of beta waves, which are associated with stress, anxiety, and concentration. There were higher peaks and a higher average stress level, plus it increased slowly as time went on.

Taking ten-minute breaks kept stress readings lower on average and prevented them from rising. And they increased other measurements of positive engagement.

Graph showing how breaks keep stress low during video calls.

Image Credits: Microsoft/Valerio Pellegrini

It’s certainly validating even if it seems obvious. And while EEG readings aren’t the most exact measurement of stress, they’re fairly reliable and better than a retrospective self-evaluation along the lines of “How stressed were you after the second meeting on a scale of 1-5?” And of course it wouldn’t be safe to take your laptop into an MRI machine. So while this evidence is helpful, we should be careful not to exaggerate it, or forget that the stress takes place in a complex and sometimes inequitable work environment.

For instance: A recent study published by Stanford shows that “Zoom Fatigue,” as they call it (a mixed blessing for Zoom), is disproportionately suffered by women. More than twice as many women as men reported serious post-call exhaustion — perhaps because women’s meetings tend to run longer and they are less likely to take breaks between them. Add to that the increased focus on women’s appearance and it’s clear this is not a simple “no one likes video calls” situation.

Microsoft, naturally, has tech solutions to the problems in its Teams product, such as adding buffer time to make sure meetings don’t run right into each other, or the slightly weird “together mode” that puts everyone’s heads in a sort of lecture hall (the idea being it feels more natural).

Stanford has a few recommendations, such as giving yourself permission to do audio only for a while each day, position the camera far away and pace around (make sure you’re dressed), or just turn off the self-view.

Ultimately the solutions can’t be entirely individual, though — they need to be structural, and though we may be leaving the year of virtual meetings behind, there can be no doubt there will be more of them going forward. So employers and organizers need to be cognizant of these risks and create policies that mitigate them — don’t just add to employee responsibilities. If anyone asks, tell them science said so.

Deep Science: Introspective, detail-oriented and disaster-chasing AIs

Research papers come out far too frequently for anyone to read them all. That’s especially true in the field of machine learning, which now affects (and produces papers in) practically every industry and company. This column aims to collect some of the most relevant recent discoveries and papers — particularly in, but not limited to, artificial intelligence — and explain why they matter.

It takes an emotionally mature AI to admit its own mistakes, and that’s exactly what this project from the Technical University of Munich aims to create. Maybe not the emotion, exactly, but recognizing and learning from mistakes, specifically in self-driving cars. The researchers propose a system in which the car would look at all the times in the past when it has had to relinquish control to a human driver and thereby learn its own limitations — what they call “introspective failure prediction.”

For instance, if there are a lot of cars ahead, the autonomous vehicle’s brain could use its sensors and logic to make a decision de novo about whether an approach would work or whether none will. But the TUM team says that by simply comparing new situations to old ones, it can reach a decision much faster on whether it will need to disengage. Saving six or seven seconds here could make all the difference for a safe handover.

It’s important for robots and autonomous vehicles of all types to be able to make decisions without phoning home, especially in combat, where decisive and concise movements are necessary. The Army Research Lab is looking into ways in which ground and air vehicles can interact autonomously, allowing, for instance, a mobile landing pad that drones can land on without needing to coordinate, ask permission or rely on precise GPS signals.

Their solution, at least for the purposes of testing, is actually rather low tech. The ground vehicle has a landing area on top painted with an enormous QR code, which the drone can see from a fairly long way off. The drone can track the exact location of the pad totally independently. In the future, the QR code could be done away with and the drone could identify the shape of the vehicle instead, presumably using some best-guess logic to determine whether it’s the one it wants.

Illustration showing how an AI tracks cells through a microscope.

Image Credits: Nagoya City University

In the medical world, AI is being put to work not on tasks that are not much difficult but are rather tedious for people to do. A good example of this is tracking the activity of individual cells in microscopy images. It’s not a superhuman task to look at a few hundred frames spanning several depths of a petri dish and track the movements of cells, but that doesn’t mean grad students like doing it.

This software from researchers at Nagoya City University in Japan does it automatically using image analysis and the capability (much improved in recent years) of understanding objects over a period of time rather than just in individual frames. Read the paper here, and check out the extremely cute illustration showing off the tech at right … more research organizations should hire professional artists.

This process is similar to that of tracking moles and other skin features on people at risk for melanoma. While they might see a dermatologist every year or so to find out whether a given spot seems sketchy, the rest of the time they must track their own moles and freckles in other ways. That’s hard when they’re in places like one’s back.

NASA makes history by flying a helicopter on Mars for the first time

NASA has marked a major milestone in its extraterrestrial exploration program, with the first powered flight of an aircraft on Mars. The flight occurred very early this morning, and NASA received telemetry confirming that the ‘Ingenuity’ helicopter it sent to Mars with its Perseverance rover. This is a major achievement, in no small part because the atmosphere is so thin on Mars that creating a rotor-powered craft like Ingenuity that can actually use it to produce lift is a huge challenge.

This first flight of Ingenuity was an autonomous remote flight, with crews on Earth controlling it just by sending commands through at the appropriate times to signal when it should begin and end its 40-second trip through the Martian ‘air.’ While that might seem like a really short trip, it provides immense value in terms of the data collected by the helicopter during the flight. Ingenuity actually has a much more powerful processor on board than even the Perseverance rover itself, and that’s because it intends to gather massive amounts of data about what happens during its flight test so that it can transmit that to the rover, which then leapfrogs the information back to Earth.

As mentioned, this is the first ever flight of a powered vehicle on Mars, so while there’s been lots of modelling and simulation work predicting how it would go, no one knew for sure what would happen before this live test. Ingenuity has to rotate its rotor at a super-fast 2,500 RPM, for instance, compared to around 400 to 500 RPM for a helicopter on Earth, because of how thin the atmosphere is on Mars, which produced significant technical challenges.

What’s the point of even flying a helicopter on Mars? There are a few important potential applications, but the first is that it sets up future exploration missions, making it possible for NASA to use aerial vehicles for future science on the red planet. It can explore things like caves and peaks that rovers can’t reach, for instance. Eventually, NASA is also hoping to see if there’s potential for use of aerial vehicles in future human exploration of Mars, too — martian explorers would benefit significantly from being able to use aircraft as well as ground vehicles when we eventually get there.