Tag: eldercare

Posted on

Toyota Research Institute SVP on the difficulty of building the perfect home robot

Earlier this week, the Toyota Research Institute opened the doors of its Bay Area offices to members of the media for the first time. It was a day full of demos, ranging from driving simulators and drifting instructors to conversations around machine learning and sustainability.

Robotics, a longtime focus of Toyota’s research division, were on display, as well. SVP Max Bajracharya showcased a pair of projects. First was something more along the lines of what one would expect from Toyota: an industrial arm with a modified gripper designed for the surprisingly complex task of moving boxes from the back of a truck to nearby conveyor belts — something most factories are hoping to automate in the future.

The other is a bit more surprising — at least for those who haven’t followed the division’s work that closely. A shopping robot retrieves different products on the shelf based on bar codes and general location. The system is able to extend to the top shelf to find items, before determining the best method for grasping the broad range of different objects and dropping them into its basket.

The system is a direct outgrowth of the 50-person robotics team’s focus on eldercare, aimed at addressing Japan’s aging population. It does, however, represent a pivot away from their original work of building robots designed to execute household tasks like dishwashing and food prep.

You can read a lengthier writeup of that pivot in an article published on TechCrunch earlier this week. That was drawn from a conversation with Bajracharya, which we’re printing in a more complete state below. Note that the text has been edited for clarity and length.

Image Credits: Brian Heater

TechCrunch: I was hoping to get a demo of the home robot.

Max Bajracharya: We are still doing some home robot stuff[…] What we’ve done has shifted. Home was one of our original challenge tasks.

Eldercare was the first pillar.

Absolutely. One of the things that we learned in that process is that we weren’t able to measure our progress very well. The home is so hard. We pick challenge tasks because they are hard. The problem with the home is not that it was too hard. It was that it was too hard to measure the progress we were making. We tried a lot of things. We tried procedurally making a mess. We would put flour and rice on the tables and we would try to wipe them up. We would put things throughout the house to make the robot tidy. We were deploying into Airbnbs to see how well we were doing, but the problem is we couldn’t get the same home every time. But if we did, we would overfit to that home.

Isn’t that ideal that you don’t get the same home every time?

Exactly, but the problem is we couldn’t measure how well we were doing. Let’s say we were a little better at tidying this one house, we don’t know if that’s because our capabilities got better or if that house was a little easier. We were doing the standard, “show a demo, show a cool video. We’re not good enough yet, here’s a cool video.” We didn’t know whether we were making good progress or not. The grocery challenge task where we said, we need an environment where it’s as hard as a home or has the same representative problems as a home, but where we can measure how much progress we’re making.

You’re not talking about specific goals to either the home or supermarket, but solving for problems that can span both of those places.

Or even just measure if we’re pushing the state of the art in robotics. Are we able to do the perception, the motion planning, the behaviors that are, in fact, general purpose. To be totally honest, the challenge problem kind of doesn’t matter. The DARPA Robotics Challenges, those were just made-up tasks that were hard. That’s true of our challenge tasks, too. We like the home because it is representative of where we eventually want to be helping people in the home. But it doesn’t have to be the home. The grocery market is a very good representation because it has that huge diversity.

Image Credits: Brian Heater

There’s a frustration, though. We know how difficult these challenges are and how far off things are, but some random person sees your video, and suddenly it’s something that’s just over the horizon, even though you can’t deliver that.

Absolutely. That’s why Gill [Pratt] says every time, ‘reemphasize why this is a challenge task.’

How do you translate that to normal people? Normal people aren’t hung up on challenge tasks.

Exactly, but that’s why in the demonstration you saw today, we tried to show the challenge tasks, but also one example of how you take capabilities that come out of that challenge and apply it to a real application like unloading a container. That is a real problem. We went to factories and they said, ‘yes, this is a problem. Can you help us?’ And we said, yeah, we have technologies that apply to that. So now we’re trying to show coming out of these challenges are these couple of few breakthroughs that we think are important, and then apply those to real applications. And I think that that’s been helping people understand that, because they see that second step.

How large is the robotics team?

The division is about 50 people evenly split between here and Cambridge, Massachusetts.

You have examples like Tesla and Figure, which are trying to make all-purpose humanoid robots. You seem to be heading in a different direction.

A little bit. Something we’ve observed is that the world is built for humans. If you’ve just got a blank slate, you’re saying I want to build a robot to work in human spaces. You tend to end in human proportions and human-level capabilities. You end with human legs and arms, not because that’s the optimal solution, necessarily. It’s because the world has been designed around people.

Image Credits: Toyota Research Institute

How do you measure milestones? What does success look like for your team?

Moving from the home to the grocery store is a great example of that. We were making progress on the home but not as fast and not as clearly as when we move to the grocery store. When we move to the grocery store, it really becomes very evident how well you’re doing and what the real problems are in your system. And then you can really focus on solving those problems. When we toured both logistics and manufacturing facilities of Toyota, we saw all of these opportunities where they’re basically the grocery shopping challenge, except a little bit different. Now, the part instead of the parts being grocery items, the parts are all the parts in a distribution center.

You hear from 1,000 people that you know, home robots are really hard, but then you feel like you have to try for yourself and then you like, really, you make all the same mistakes that they did.

I think I’m probably just as guilty as everybody else. It’s like, now our GPUs are better. Oh, we got machine learning and now you know we can do this. Oh, okay, maybe that was harder than we thought.

Something has to tip it at some point.

Maybe. I think it’s going to take a long time. Just like automated driving, I don’t think there’s a silver bullet. There’s not just like this magical thing, that’s going to be ‘okay, now we solved it.’ It’s going to be chipping away, chipping away, incrementally. That’s why it’s important to have that kind of roadmap with the shorter timelines, you know, shorter or shorter milestones that give you the little wins, so you can keep working at it to really achieve that long-term vision.

What’s the process for actually productizing any of these technologies?

That’s a very good question that we are ourselves trying to answer. I believe we kind of understand the landscape now. Maybe I was naïve in the beginning thinking that, okay, we just need to find this this person that we’re going to throw the technology over to a third party or somebody inside of Toyota. But I think we’ve learned that, whatever it is — whether it’s a business unit, or a company, or like a startup or a unit inside of Toyota — they don’t seem to exist. So, we are trying to find a way of creating and I think that’s the story of TRI-AD, a little bit as well. It was created to take the automated driving research that we were doing and translate into something that was more real. We have the same problem in robotics, and in many of the advanced technologies that we that we work on.

Image Credits: Brian Heater

You’re thinking about potentially getting to a place where you can have spinoffs.

Potentially. But it’s not the main mechanism by which we would commercialize the technology.

What is the main mechanism?

We don’t know. The answer is the diversity of things that we’re doing is very likely going to be different for different groups.

How has TRI changed since its foundation?

When I first started, I feel like we were very clearly just doing research in robotics. Part of that is because we were just so very far away from the technology being applicable to almost any real-world challenging application in a human environment. Over the last five years, I feel like we’ve made enough progress in that very challenging problem that we are now starting to see it turn into these real-world applications. We have consciously shifted. We’re still 80% pushing the state of the art with research, but we’ve now allocated maybe 20% of our resources to figuring out if that research is maybe as good as we think it is and if it can be applied to real-world applications. We might fail. We might realize we thought we made some interesting breakthroughs, but it’s not anywhere near reliable or fast enough. But we’re putting 20% of our effort toward trying.

How does eldercare fit into this?

I would say, in some ways, it’s still our north star. The projects are still looking at how we ultimately amplify people in their homes. But over time, as we pick these challenge tasks, if things trickle out that are applicable to these other areas, that’s where we’re using these short-term milestones to show the progress in the research that we’re making.

How realistic is the possibility of a fully lights-out factor?

I think if you were able to start from scratch in maybe in the future, that might be a possibility. If I look at manufacturing today, specifically for Toyota, it seems very unlikely that you could get anywhere close to that. We [told factory workers], we’re building robotic technology, where do you think it could apply? They showed us many, many processes where it was things like, you take this wire harness, you feed it through here, then you pull it out here, then you clip it here, and you clip it here, and you take it here, and you take it here, and then you run it like this. And this takes a person five days to learn the skill. We were like, ‘yeah, that’s way too hard for the robotic technology.’

But the things that are the most difficult for people are the ones you would want to automate.

Yes, difficult or potentially injury prone. For sure, we would like to make stepping stones to get to that eventually, but where I see robotic technology today, we’re quite far away from that.

Toyota Research Institute SVP on the difficulty of building the perfect home robot by Brian Heater originally published on TechCrunch

Posted on

Toyota Research Institute’s robots leave home

“I think I’m probably just as guilty as everybody else,” Toyota Research Institute’s (TRI) senior vice president of robotics, Max Bajracharya, admits. “It’s like, now our GPUs are better. Oh, we got machine learning and now you know we can do this. Oh, okay, maybe that was harder than we thought.”

Ambition is, of course, an important aspect of this work. But there’s also a grand, inevitable tradition of relearning mistakes. The smartest people in the room can tell you a million times over why a specific issue hasn’t been solved, but it’s still easy to convince yourself that this time — with the right people and the right tools — things will just be different.

In the case of TRI’s in-house robotics team, the impossible task is the home. The lack of success in the category hasn’t been for lack of trying. Generations of roboticists have agreed that there are plenty of problems waiting to be automated, but thus far, successes have been limited. Beyond the robotic vacuum, there’s been little in the way of breakthrough.

TRI’s robotics team has long made the home a primary focus. That’s driven, in no small part, by it choosing eldercare as a “north star” for the same reason that Japanese firms are so far ahead of the rest of the world in the category. Japan has the world’s highest proportion of citizens over the age of 65 — trailing only Monaco, a microstate in Western Europe with a population of fewer than 40,000.

In a world where our health and wellness are so closely tied to our ability to work, it’s an issue bordering on crisis. It’s the kind of thing that gets Yale assistant professors New York Times headlines for suggesting mass suicide. That’s obviously the most sensationalistic of “solutions,” but it’s still an issue in search of meaningful solution. As such, many Japanese roboticists have turned to robotics and automation to address issues like at-home healthcare, food preparation and even loneliness.

Image Credits: Brian Heater

Early, professionally produced videos showcased robotics in the home, executing complex tasks, like cooking and cleaning a broad range of surfaces. When TRI opened the doors of its South Bay labs to select press this week to show off a range of its different projects, the home element was notably lacking. Bajracharya showcased a pair of robots. The first was a modified off-the-shelf arm that moved boxes from a pile onto nearby conveyer belts, in a demo designed for unloading trucks — one of the more difficult tasks to automate in an industrial warehouse setting.

The second was a wheel robot that goes shopping. Unlike the warehouse example, which had standard parts with a modified gripper, this system was largely designed in-house out of necessity. The robot is sent out to retrieve different products on the shelf based on bar codes and general location. The system is able to extend to the top shelf to find items, before determining the best method for grasping the broad range of different objects and dropping them into its basket. The system is an outgrowth of the team’s pivot away from home-specific robots.

Image Credits: Brian Heater

To the side of both robots is a mock kitchen, with a gantry system configured to the top of its walls. A quasi-humanoid robot hangs down, immobile and lifeless. It goes unacknowledged for the duration of the demos, but the system will look familiar to anyone who has watched the team’s early concept videos.

“The home is so hard,” says Bajracharya. “We pick challenge tasks because they are hard. The problem with the home is not that it was too hard. It was that it was too hard to measure the progress we were making. We tried a lot of things. We tried procedurally making a mess. We would put flour and rice on the tables and we would try to wipe them up. We would put things throughout the house to make the robot tidy. We were deploying into Airbnbs to see how well we were doing, but the problem is we couldn’t get the same home every time. But if we did, we would overfit to that home.”

Moving into the supermarket was an effort to address a more structured environment while still tackling a pressing issue for the elderly community. In testing the product, the team has moved from Airbnbs to a local mom-and-pop grocery store.

Image Credits: Brian Heater

“To be totally honest, the challenge problem kind of doesn’t matter,” Bajracharya explains. “The DARPA Robotics Challenges, those were just made up tasks that were hard. That’s true of our challenge tasks, too. We like the home because it is representative of where we eventually want to be helping people in the home. But it doesn’t have to be the home. The grocery market is a very good representation because it has that huge diversity.”

In this instance, some of the learnings presented in this setting do translate to Toyota’s broader needs.

What, precisely, constitutes progress for a team of this nature is a difficult question to answer. It’s certainly one that’s top of mind, however, as large corporations have begun cutting roles in longtail research projects that have yet to deliver tangible, monetizable results. When I put the question to Gill Pratt yesterday, the TRI boss told me:

Toyota is a company that has tried very hard not to have employment follow business cycle. The car business is one that has booms and busts all the time. You may know that the history of Toyota is to try not to lay people off when times are tough, but instead go through a couple of things. One is shared sacrifice, where people take up the cause. The second is to use those times to invest in maintenance, plans and education to help people get trained.

Image Credits: Brian Heater

Toyota is well-known in the industry for its “no layoffs” policy. It’s an admirable goal, certainly, especially as companies like Google and Amazon are in the midst of layoffs numbering in the tens of thousands. But when goals are more abstract, as is the case with TRI and fellow research wings, how does a company measure relevant milestones?

“We were making progress on the home but not as fast and not as clearly as when we move to the grocery store,” the executive explains. “When we move to the grocery store, it really becomes very evident how well you’re doing and what the real problems are in your system. And then you can really focus on solving those problems. When we toured both logistics and manufacturing facilities of Toyota, we saw all of these opportunities where they’re basically the grocery shopping challenge, except a little bit different. Now, instead of the parts being grocery items, the parts are all the parts in a distribution center.”

As is the nature of research projects, Bajracharya adds, sometimes the beneficial outcomes are unexpected: “The projects are still looking at how we ultimately amplify people in their homes. But over time, as we pick these challenge tasks, if things trickle out that are applicable to these other areas, that’s where we’re using these short-term milestones to show the progress in the research that we’re making.”

The path toward productizing such breakthroughs can also be fuzzy sometimes.

“I believe we kind of understand the landscape now,” Bajracharya. “Maybe I was naive in the beginning thinking that, okay, we just need to find this person that we’re going to throw the technology over to a third party or somebody inside of Toyota. But I think what we’ve learned is that, whatever it is — whether it’s a business unit, or a company, or like a startup or a unit inside of Toyota — they don’t seem to exist.”

Spinning out startups — akin to what Alphabet has done with its X labs — is certainly on the table, even though it isn’t likely to be the primary path toward productization. What form that path will ultimately take, however, remains unclear. Though robotics as a category is currently far more viable than it was when TRI was founded in 2017.

“Over the last five years, I feel like we’ve made enough progress in that very challenging problem that we are now starting to see it turn into these real-world applications,” says Bajracharya. “We have consciously shifted. We’re still 80% pushing the state of the art with research, but we’ve now allocated maybe 20% of our resources to figuring out if that research is maybe as good as we think it is and if it can be applied to real-world applications. We might fail. We might realize we thought we made some interesting breakthroughs, but it’s not anywhere near reliable or fast enough. But we’re putting 20% of our effort toward trying.”

Toyota Research Institute’s robots leave home by Brian Heater originally published on TechCrunch

Posted on

Labrador Systems deploys its first assistive elder-care robots

We’ve been keeping tabs on Labrador Systems since we caught a very early demo of its elder care-focused technology in a hotel suite several CESes ago. Today the California-based robotics firm announced that it’s begun deploying its Retriever Pro system to a handful of early clients, including, On Lok PACE, Nationwide Insurance, Masonic Homes of California, Western Homes Communities, Eskaton, The Perfect Companion, Presbyterian Villages of Michigan, University of Michigan Flint and Graceworks Lutheran Services.

The news follows extended piloting for the system in places like senior living communities. The Retriever Pro is designed to bring a kind of assistive freedom to people living on their own with mobility limitations. It’s a clever technology that effectively amounts to a semi-autonomous mobile shelving system that can be used to deliver objects they might otherwise have issues carrying.

“The burden on caregivers is growing at a rate that is simply not sustainable. Organizations are already experiencing major caregiver shortages, and in the coming years there will be significantly more people in my parents’ age group (85+) with fewer people to help take care of them,” CEO Mike Dooley said in a release tied to the news. “Our mission is to provide relief on both sides of that equation, empowering individuals who need care to do more on their own while extending the impact of each caregiver’s visit well beyond the time they are physically present.”

Image Credits: Labrador

The world of elder-care robotics is still fairly nascent in the U.S. Japan may have the largest head start in the category due, in part, to its aging population, but the concept has been growing in acceptance. A number of firms working to design more all-purpose systems have pointed to living assistance as a potential application, but currently the robotic market isn’t exactly flush with this tech.

The company says it also “continues to move forward with development and testing” of its more consumer-focused system, the Retriever. Dooley clarified the difference between the two in a comment to TechCrunch, noting:

The key added features for the Pro are for bringing caregivers and staff into the loop and overall supporting the care provider on their mission.  A portion of that is on the software side, with integration with enterprise grade solutions for care management. So for example, caregiving organizations could have multiple users log on to set schedules for the robot, check activity reports and remotely assist with the robot operation. On the hardware side, we’d have more options for carrying and powering a 3rd party tablet or other screen device that the care organization may already be using, to move that device through the home. The Pro will also have provisions for supporting cellular connectivity as an upgrade.

Labrador set to deliver a robotic helping hand to homes in 2023

Labrador Systems deploys its first assistive elder-care robots by Brian Heater originally published on TechCrunch

Posted on

The ElliQ eldercare robot is finally available

The astute TechCrunch reader will quickly note that we’ve been covering Intuition Robotics for five years now, dating back to the eldercare robotics’ crowdfunding campaign way back in February 2017. Most of the coverage since then has found the Israeli company raising even more money across various rounds, without actually answering the most important question: when? Specifically, when is the ElliQ robot going on sale?

Seems the product is finally ready for prime time. What can I say, robots take a long time, and the company has spent several years beta testing. Intuition just announced that ElliQ is officially available today, through the product’s site. As is the wont of the robotics industry in 2022, the device will be available through a subscription plan — RaaS, if you will. That starts with a $250 upfront fee and then runs $30 a month (if you go in for an annual subscription).

Eldercare has long been a key robotics focus in Japan, but has had issues gaining a foothold elsewhere. Here in the States, a smattering of startups are getting on board, including Labrador Systems, while smart home device makers like Amazon and Google have begun building related functionality into their systems.

Image Credits: Intuition Robotics

I tend to think of ElliQ as something along the lines of the sadly discontinued Kuri or Jibo, focused specifically for an older audience. Rather than actually assisting with the chores like Labrador, it’s designed to keep older adults engaged. The company says average users engage with the product 20 times a day for a combined 20 minutes or so. Meaning, it’s not designed for the engagement level, of say, TV, but rather something more along the lines of frequent, brief check-ins.

“After years of hard work this day has finally come,” co-founder and CEO Dor Skuler said in a release. “Over the course of the pandemic, we’ve seen the devastating effect that loneliness can have on the older adult population. At the same time, we’ve seen ElliQ be incredibly helpful to our beta users and put a smile on their faces.”

There’s a range of content available through the device, including workouts, health info from the Mayo Clinic, check-ins with loved ones and transportation through Uber Health, among other things. Effectively, assistance and engagement are the two things here for people who are independent enough to live on their own, but require a bit more help/support.

Posted on

This lightbulb can monitor your vital signs

Here’s a fun left-field idea from CES this week: a smart lightbulb capable of taking health readings, including heart rate, temperature and tracking sleep. Necessary? Not in the slightest. Interesting, yeah, sure.

On the face of it, the product operates in a similar capacity as the latest Nest Hub — and uses a similar technology. Radar sensing is at the heart of it, measuring changes in the user’s body to deliver a kind of passive health monitoring system — i.e. one that doesn’t require you to put on a smartwatch or fitness band.

Of course, there are a lot more variables with a lightbulb, versus a product specifically designed to sit next to your head while you sleep. There are also some broader questions here, like, is the sense of convenience in not wearing a fitness band — or even ring — so great that you’d want to outsource it to a lightbulb? And what of accuracy? It’s tough to compete with readings from sensors mounted directly next to the body.

The smarthome gadget has Bluetooth and Wi-Fi functionality, and could have some potentially useful applications for eldercare, including fall detection. The Verge has a bit more insight into the bulb, which is set to launch toward the end of the year for an undisclosed price. Sengled announced a number of other products at the show, as well, including a smart oil diffuser, a portable lamp and a motion sensor.

Read more about CES 2022 on TechCrunch

Posted on

Labrador set to deliver a robotic helping hand to homes in 2023

Back at CES 2020, Labrador Systems co-founder/CEO Mike Dooley told me, “I think there are fewer fake robots [at the show] this year.” Over the past several years, we’ve seen the show begin to treat robotics less as novelty and more as serious home devices. At the time, the company had rented a suite where it was showing off an early version of the system it would come to call, fittingly, Retriever.

Early last year, following the first all-virtual CES, the company began rolling out prototypes to select users. At this week’s show, we’re getting to see the fruits of those early tests, in the form of testimonial videos. Retriever is effectively a more refined version of what I saw two years back, but essentially works on the same principle. It resembles a robotic bar cart, offering assistance for elderly users and people with limited mobility.

Image Credits: Labrador Systems

If you’ve been following the industry, you no doubt know that eldercare has been a big business for robotics in Japan, due to an aging population. That’s been less of a case here in the States, though companies are starting to follow suit. Retriever is one of the best examples I’ve seen thus far of an American company purpose-building a robot to assist people in this way. The product is primarily aimed at those people who are capable of living independently, but could benefit from an added robotic hand.

“There’s a significant portion of our society that’s massively underserved,” Dooley said in a release tied to the news. “When pain or other health issues start interfering with your ability to move yourself or other things, even short distances can have a major impact on your independence, quality of life and overall health. The Retriever is meant to help physically bridge some of that gap and empower individuals to be more active and do more on their own.”

Image Credits: Labrador Systems

The system is capable of carrying up to 25 pounds, and can be used to deliver laundry, meals and other payloads around the house. It features a retractable tray system that can move objects onto the cart from counters, shelves and a bespoke refridgerator the company says it’s planning to offer. Underneath that is an additional storage space for things like food and medication, as well as a port for charging phones.

The system will also feature voice control by way of Alexa — fitting, given that the Amazon Alexa Fund was one of the startup’s backers (along with SOSV/HAX, iRobot and the National Science Foundation). Following an upcoming beta test period, Labrador plans to hit commercial sales by the second half of next year, at a range of different price points, based on functionality. Early adopters can secure a Retriever unit for $1,500 up front, plus a monthly services fee of between $99 and $149 a month, depending on financing.

The company is also using the opportunity to announce a $3.1 million seed round co-lead by Alexa Fund and iRobot Ventures. The funding will go toward increasing headcount for engineering and accelerating production.

“Labrador is advancing the state of the art in what it means to provide assistance to people aging and with mobility challenges,” Alexa Fund’s Paul Bernard said in a statement tied to the news. “They are addressing a significant problem in our society and have brought their decades of experience in consumer robotics to bear, delivering a product that will help empower people to live better lives.”

Read more about CES 2022 on TechCrunch