A Case Study Interview with the University of Sheffield

As the pandemic struck universities across the UK in the 20/21 academic year, they had to quickly adjust to teaching students remotely. The Diamond at Sheffield University was able to deliver more than 2,700 hours of computer science teaching to over 300 students. Read on to learn about the learnings and challenges of teaching computer science remotely and the benefits of using MiRo and MiRoCLOUD.

David Lane (L), Professor Emeritus David Lane CBE FREng FRSE, Chairman Consequential Robotics and co Founder of the Edinburgh Centre for Robotics and National ROBOTARIUM at Heriot-Watt and Edinburgh Universities interviews Alex Lucas (M), Lecturer at the Department of Computer Science at Sheffield University and Tony Prescott (R), Professor of Cognitive Robotics and Chief Scientist at Consequential Robotics.

Teaching in a Pandemic

Alex, how you have been teaching various level one courses in computer science including using MiRo and MiRoCLOUD to teach programming?

In the department of computer science, for the first-year level one teaching and in particular for the module on machines and intelligence, we use MiRoCLOUD quite extensively to illustrate some of the concepts from the module we teach. In the first part of the module, we teach agent architectures and intelligence agents starting from simple reflex agents model based goal-based and so on. In the second part, we move on to looking at computer vision and how machines do their vision. We have been able to use MiRo’s and MiRoCLOUD to do some monitoring detection in a condition, both in the simulation and in the real world. 

The students were able to interact with the robots so they could have a hands-on experience and play around and have a look at them. Obviously, they were not able to do this during the pandemic, so we had to adapt. We created a set up where we had the demonstrators handling the robots. During the face-to-face sessions students came in and socially distanced. They used MiRoCLOUD to work in the simulator and when they were ready to put their code on the robot, they just connect to the robot.

Students did this in the arena, but if they wanted to move to a specific location, they asked the demonstrators to do so and they can switch between the simulator and the MiRo robot quite easily. We have projectors where we stream the viewing camera, and it shows what is happening in the arena for the students who are quite far away and cannot see. 

Whenever and wherever you are in the world, you can enter Sheffield University and then, and there, when you watch MiRoCLOUD, you can access the robots that are in the room. You can both see what is going on in the arena and control the robot at the same time, virtually anywhere, which is quite cool. For example, students were trying to do some ball detection by trying to lock on the ball and kick it off of the arena. We also use MiRo for other kinds of teaching as well as throughout the curriculum. 

Alex, to what extent were the students working at home and working in the lab?

We obviously cannot accommodate all 300 students with the social distancing, so we had to do slots where students sign up. We usually had about 80 students per week, and the rest were connecting remotely, both during the session. We had about two or three sessions, and the remote students could connect to those as well. The students worked in groups of four. The students who are in a group but were connecting remotely could work and collaborate with the students who are in the class and share the goal and see what is going on. We also had another just an online session on a different day, but all students can connect remotely and ask questions, help and support.

Alex, are all the students able to code when they start the course?

With computer science degrees in the UK, it is usually not a requirement that applicants have computer science ‘A’ level so we get a real mixed range of programming abilities. Some people don’t know programming and some people are experts at it, so it is an opportunity to bring people up to speed on their programming. In another course, we teach people about algorithms using a MiRo and the Google Blockly, which is a visual programming language that really anybody can get up to speed with in just a few minutes. I know that in some of the schools where MiRo was being used, the pupils learn to program using Blockly on their own and then they go and play with the more complicated coding to try and figure it out.

The good thing about MIRoCLOUD is you can start off with very simple coding in the Blockly interface, which is a very intuitive interface. The blocks are preset with certain actions and movements and some math and logic. It's very quick and easy as a teacher to dive into that. I give instructions on what the blocks do, and basically that's all the students need to start going. Once we are into the second part of the core of the module, we switch on to python. This is a very unique fixture to MIRoCLOUD, where you can transform the block, the code in Python convert quite nicely. You don't actually have to know that much of Python to be able to program in Python because you can start off in Blockly and transform the code to Python, and then just make small changes to the Python code if you want to really fine tune something. The visual programming language is great for getting going, but then once you become a bit faster at it, you want to be able to get in the parameters quickly and the Python's good for that. We also did an open day session for some of the applicants in the summer with MIRoCLOUD and then an intro week session as well.

Alex, how do you swap between the simulator to the robot?

Click a button to swap between the simulation and the actual robot. Students can share robots because you simply have to enter the robots IP address into MiRoCLOUD so students can take it in turns to use that IP address. It also means that you don't actually have to be in the room to connect to the robot, you could potentially be at home. Since we have videos, students who can't be in the lab can watch via a video and see what the robot is doing and also send a program and have it run on the robot.

Alex, how did you use MiRo for computer vision?

We have done lots of different projects using computer vision. On the simulation screen in MiRoCLOUD, you can see what MiRo sees in the bottom right corner which are the actual views from the two cameras used for MiRo’s eyes. They are high-resolution cameras. You get quite a good image and because you have got two cameras and there is some overlap, you could potentially extract that through stereo which gives you distance as well. You can do a lot with the computer vision without actually having the robot. But of course you could then put it on the actual robot and control your robot to move around.

All the sensors that are on MiRo, as well as its actuators, and with all the open-source software that is around, you can really build in quite a lot of power. Just the power of open-source software that can be integrated onto MiRo and into the MiRoCLOUD environment means that the sky's the limit. It is all about the creativity of the students and the teaching staff as well.

Tony, how have you been using MiRo to teach level three courses?

Alex and I are teaching a level three course called cognitive and biomimetic robotics. MiRo is a biomimetic robot designed to look a bit like an animal, but more importantly, it is designed to think a bit like an animal. Its control architecture is modeled at a very high level of abstraction on how the brain works so we teach some of that in the level three course. I think that illustrates how robotics can be used to teach concepts in biology, as well as concepts in computer science. We can tell students a bit about how brains work as well by showing them how this robot is programmed. We also use these robots for undergraduate projects, so we have level three students doing projects and myself or Alex or other supervisors supervising.

It is usually a project you do over the course of two semesters. You go into quite a lot of depth and often install the full MiRo developer kit on your own computer for that, which gives you a lot of flexibility about how you program it. We do group projects as well, for example, robot football. This is from a couple of years back and I don't know how much you know about the state of the art in robot football, but robots aren't very good at football and MiRo is certainly a bit of a beginner. This was the first time we really tried to do robot football. A couple of students managed to program the two robots to do a corner kick. We also had one student who did a project on reinforcement learning, using stereo vision, and that was published at a paper and a robotics conference and was nominated for a best paper award.

All, what have we learned about teaching from the pandemic?

David: That we are probably going to keep doing what we have had to adjust to and because it is good for the way that students learn. Challenges around how we introduce creativity and practical work remotely at scale. For example, recently supporting 300 students from China who were on the VPN. We want to map out how we teach in the future and not just for young students in universities, but for people throughout their lives who want to learn new skills and retrain.

Tony: These sorts of facilities are going to be really important. Nowadays you go to a library and borrow a book, but I think in the future, maybe you'll go to the university library or the local library and borrow a robot. And you know, this is what we'll need in order to bring up the skills of the wider population so that we can do the jobs of the future. We need to be able to have robots that people can access easily and learn how to use them. And I think that could be one way of doing it. Borrowing a robot.

Alex: We have actually been piloting the scheme before the pandemic hit. We had lockers in the Diamond Centre where we have 10 robotics kits, the MiRo, the laptop. Anyone who was interested and involved in these courses could come in and borrow them for a limited time.

Tony, what makes MiRo suitable for this blended learning (online and offline)?

MiRo is a mobile robot with lots of sensors on it and it works out of the box. If you're teaching a class with 40 robots, it could take a lot of time if you have to build them from scratch, but you don't have to assemble MiRo. It has a lot of the things that you would want in a teaching robot in terms of sensing and actuation. So, lots of ways of sensing the world, particularly high-quality stereo vision, different microphones, touch sensing and then also different ways of moving. It can drive around, move its head, and move its eyelids in an expressive way. It is low cost, so we have loaned them out to students. They have been able to take them back to the halls of residence, which was particularly helpful during the lockdown.

The most unique thing about MiRo is that we've developed the simulation environment - MiRoCLOUD, which uses a full physics simulator. You can see first what a MiRo would do in the real world. Of course, the real was never quite like simulation, but you can get close. You can do computer vision in the simulation environment and various other forms of sensing. And with action, you can move, bump into things and so on, you can fall off tables and so on. MiRoCLOUD, runs entirely in the cloud so no installation process is required.

The process of installing on a computer was really a turnoff for education establishments. It can get very complicated because you have lots of different computer platforms and a complex installation process, lots of software that you potentially need and security issues. We've tried this in Sheffield before and it has been difficult. We also tried using existing cloud-based solutions. They generally don't have enough bandwidth power to run simulated robots so we developed our own solution with Amazon Web Services. What we have now is the scalable web cloud-based programming environment where you can simulate the robot pretty much in real time and run your own programs. We've been rolling that out as Consequential Robotics since the summer and fortunately for the University of Sheffield, it coincided with taking their teaching online. 

By using Amazon web services, MiRoCLOUD users have all that computing power to support good modeling and to be able to work remotely and then of course, download to the actual robot. This will also be useful in a practical teaching environment. We have also found that it is really essential to have the multi-user capability. With over 300 students in year one and half of them are working on MiRoCLOUD at the same time, that then requires a lot of computer processing to simulate a three-dimensional world and interaction. Most university workstations can more or less support it, but a lot of student laptops and devices won't work with that. So the advantage is that students can just open a browser window and log into their account with access to a fast simulation. Everyone has the same experience.