Student Showcase: Stimulating Psychological Behaviour in MiRo

Our Student Showcase blog series highlights some of the amazing student projects utilising MiRo. These articles are written by the students about their projects and this piece is written by Aung Htet from the University of Sheffield.

---

Introduction

My name is Aung Htet, and I am currently working on the MiRo to produce a simulation of a psychological experiment called the Strange Situation. It is a continuation of a group project I worked on while at the University of Sheffield. Being allowed to continue on this project along with my supervisor, Dr. Alejandro Jimenez-Rodriguez, I have been working on this project at Sheffield Robotics to be able to embody the simulation and write a paper related to it. The idea of the project is to be of use in affective and social robotics, where it will allow the MiRo to simulate behaviours rather than following a set of instructions that have been tasked for completion.

What the project is about

As an introduction to the project, it is based on a psychological experiment called the Strange Situation Procedure, which is based on the relationship between the caretaker and the child. The purpose of this experiment is to observe different attachment behaviours between the mother and child, which can be exhibited in different scenarios. The possible attachment styles can either be secure, ambivalent or avoidant.

Thus, in our experiment for simulating behaviour, we will be simulating these three types of behaviour by having the MiRo decide whether to explore or care for each other. The decision process will be made through a coupled oscillator dependent on the ambivalent level, avoidant level and the information inferred from one another. There are two parts to the project: first, to allow the MiRos to directly infer from each other, followed by the MiRos having the ability to take cues for inferring information.

Progress on what is being done or what will be done

The first step of the project involves using a QR code to determine where the MiRos would meet, and sound recognition will be involved in determining whether care is being given or received. In this step, the parent and child share a common state that would be communicated to all the nodes. The step has been successful, with a demonstration being done at the Sheffield Robotics lab.

First stage of the project

Second stage of the project

The second step of the project would involve the MiRo to determine each other's state through inference on cues like sounds and gestures.

Thus, further improvements will be made concerning the detection of audio through speech recognition and detecting the “coo” of another MiRo. This will be used on communications between the MiRos with current progress being done on speech recognition. Currently, the MiRo is focused on being able to detect a few commands, which set the MiRo to do some actions.

The MiRo will also be able to detect each other through image recognition rather than make use of a QR code to allow proper simulation of detecting the other MiRo and care being given. This will allow the MiRo to meet each other rather than setting a rendezvous point through the use of a QR code.

POST AUTHOR

Aung Htet

Computer Science Student at The University of Sheffield

Student Showcase: Interacting with MiRo via Auditory-Mediated Sensations

The start of our Student Showcase blog series which highlights some of the amazing student projects utilising MiRo. These articles are written by the students about their projects and this piece is written by Logan Miller from the University of Sheffield.

---

During my 3rd year at the University of Sheffield my team and I were tasked with adding human-robot interactions to the MiRo robot. The MiRo has a soft tail and ears which produce a specific crunchy noise when pinched or stroked. We planned to use the four microphones present in the MiRo’s body to detect the noise from stroking or pinching these body parts. This could then be used to train the robot, such as stroking to reward it and pinching to punish it – a common machine learning technique called reinforcement learning.

To detect if MiRo has been stroked or pinched, and where it happened, we used a classifier. A classifier is a form of supervised learning which uses labelled training data to map the various sounds to the actions that produced them.

We used the MiRo’s four microphones to record five minutes of us stroking the tail and ears, followed by five minutes of us pinching them and then five minutes of background noise in our lab. After recording our training data, we used Google’s VGGish library in Python to convert our audio files into semantically meaningful high-level 128-dimension embed files which produced better results when fed into a classification model compared to the audio files.

We then mapped each 128D embed file from the training data to a corresponding action and trained a classification model using a linear support-vector machine SVM classifier from the Ski-Kit Learn Python library. As our project focused on detecting the pinch or stroke the MiRo responded with either a blue, green or red LED for if it detected background noise, stroking, or pinching respectively.

Images of MiRo demonstrating the three LED colour results

Background noise can cause a lot of errors in audio classification, so removing the background noise from the pinching and stroking recording would improve the classifier’s results. One way of achieving this is a noise gate, which defines a specific sound level that audio can pass. If this level is not met, no audio signal is recorded (Hodgson, 2010).

We opted to use a slightly more advanced technique where the noise gate is adapted into a spectral noise gate. This is where a signal is split into individual frequencies, with custom thresholds set for each frequency. A noise gate is then applied to each frequency, resulting in a more tailored approach depending on the situation (Audacity, 2021).

We used Tiny Machine Learning (tinyML) techniques to allow our detection to run on the low-powered Raspberry Pi 3B+ inside the MiRo. This was to improve the speed MiRo could respond to pinches and strokes compared to if our detection was running of an external machine communicating with MiRo over a WiFi network. The main objective in tinyML is to reduce the pre-trained models, which is achieved through quantization, pruning and fusing.

Quantization its often the most effective technique and reduces the accuracy of values by using fewer bits. PyTorch’s quantization functionality can reduce the model up to four times its original size by using 8-bit integers over floating point numbers (Quantization, n.d.).

Pruning is a technique which eradicates useless parameters. First a pruning criterion has to be chosen, which assigns a rank to a set of elements to be pruned. The main pruning methods are random pruning, where elements are ranked in a random order, and magnitude-based pruning, where weights below a certain threshold are set to zero.

Finally, fusing combines multiple operators together, however we decided not to implement it as there is very little performance difference in fused and non-fused models with small models, such as this one. (PyTorch Fusing, n.d.).

Overall, this was a really fun project which allowed my team and I to get hands on experience with robots. It also gave us a greater understanding of the theoretical biomimetics we learnt alongside this project – which is the combination of analysis (investigating and understanding complex biological systems), and synthesis (building the system to test our scientific understanding).

POST AUTHOR

Logan Miller

Computer Science Graduate BSc from The University of Sheffield

---

Hodgson, J. (2010). Understanding Records: A Field Guide To Recording Practice. Bloomsbury Academic.

Alternative Noise Reduction Techniques. (2021, November 16). Audacity Manual. Retrieved May 12, 2022, from https://manual.audacityteam.org/man/alternative_noise_reduction_techniques.html

Quantization — PyTorch 1.11.0 documentation. (n.d.). PyTorch. Retrieved May 12, 2022, from https://pytorch.org/docs/stable/quantization.html

Fuse Modules Recipe — PyTorch Tutorials 1.11.0+cu102 documentation. (n.d.). PyTorch. Retrieved May 12, 2022, from https://pytorch.org/tutorials/recipes/fuse.html