Professor of Computer Science Lynne Baille and her team at Heriot-Watt University have been investigating the use of MiRo for a fall alert. Read on to learn about their approach for human-centered robot interaction (HRI) research.
The significance of a fall alert
A significant portion of the over 65 population experience a fall at least once a year in their home environment. This puts a faller under significant health risks, causing distress, pain, injury, and death to the people who experience the falls and adds to the financial burden of the care services.
Almost half of fallers are unable to get up without assistance even if they have not been injured during the fall. In some cases, people experiencing a “long-lie” (more than 60 minutes) did not activate an alarm, even though they had it with them, either because they forgot that they had an alarm, or they did not want to ‘bother anyone’ by calling for an ambulance thinking they could eventually get up by themselves. Fallers also often restrict their activities due to the fear of falling. Therefore, falls affect people mentally as well as physically.
Using a social robot for assessing and evaluating the situation
The main objective is not to detect falls, but to use a social robot for assessing and evaluating the situation immediately after a fall. The first goal is to investigate if a robot could assess and monitor a post-fall situation and provide the same benefits if it is equipped with the functionality of alerting a carer. The second, is to learn how health professionals and housing association representatives will respond to the idea of using small social robots for interacting with fallers in their home environment.
Heriot-Watt University chose to use MiRo because of its small size and ‘generic mammalian’ form that resembles a cross of a number of domestic animals (rabbit, cat and dog). Its small size means MiRo will be at the fallers eye level after a fall and will behave as a pet would; arriving to its ‘master’ to see if they are OK. This may also help to promote its acceptance in the home environment for the purpose of monitoring, as robots that look and behave in a way similar to living beings increases their acceptability.
MiRo interacts with the faller
Within a controlled lab setting, the focus is on the interaction with MiRo, starting from the moment a fall is detected. In this scenario, a researcher plays the person who falls (the faller) and another researcher manually navigates the MiRo to them. Once the faller is located, MiRo turns its head and angles its neck to get the best possible view of the faller through two cameras mounted behind its eyes. It then proceeds to ask them to acknowledge they are conscious. This acknowledgement interaction can happen in a variety of different modalities; the faller can either reply vocally to MiRo saying they are “OK”, or they can touch MiRo’s body – MiRo is equipped with capacitive sensors running its entire body length and top of its head.
MiRo tries to ascertain if the faller is moving
This is done using computer vision algorithms implemented in OpenCV. These algorithms take every frame generated by the MiRo’s camera and compares it to the next frame. Differences between subsequent images are rendered in contours, and for each difference found, a rectangle is superimposed upon the contours. In order to only select big movements, like those generated by a person moving, only contours of a certain size are considered. If they are moving, MiRo continues to monitor until they get up and logs it as a ‘fall but successful recovery’. However, if they are not moving, or if they do not respond to MiRo’s earlier request for interaction, a notification is sent to the appropriate responder.
Further research opportunities
Initial informal demonstrations to health professionals and housing association representatives indicate that it has the potential of higher acceptance than CCTV cameras since you are ‘fully aware when it is looking at you’ - like a pet animal would – and being an ‘animal’ may appear less judgmental than a humanoid robot.
However, small socially assistive robots have not yet been used for helping people in distress in their home environment by being active social agents – asking the person if they are OK, and monitoring them until they are in a ‘safer state’ (i.e. back on their feet in this case) so there is an opportunity to explore this further. It would also be interesting to investigate how being monitored through a robot that looks like a small pet compares against humanoid robots, or even against other conventional monitoring installations such as CCTV.
In the future, the computer science team at Heriot-Watt also plan to implement automatic navigation, allowing MiRo to navigate a changing home environment to reach the faller. Using relatively low-resolution sensors, such as the ones currently found on the MiRo1 has been successfully used in the past for navigation and human detection purposes. The next step is to then integrate their system with existing fall detection systems before trialling their new system in a real assistive living environment, first with simulated fallers before moving on to homes of people who are at risk of falling, giving this little robot a big task.
Read the full details of the research here.
Stay tuned to our blog for more news about this research project.