Francisco Roldan Sanchez  is a PhD candidate in Insight at DCU, working in a multi-disciplinary cross-university group created with Insight colleagues in UCD. This group’s research focuses on using artificial intelligence, specifically reinforcement learning algorithms, to make robots learn manipulation skills. In particular, Francisco’s main focus is on making these algorithms more efficient with the aim of making robots learn faster new manipulation skills.

Robotic manipulation already has a wide range of applications: manufacturing and logistics, healthcare, space exploration… However, with the inclusion of artificial intelligence, we can expect to see even more applications in the future involving robots that require a deeper understanding of the environment that surrounds them, such as assistive robotics or disaster response.

Current state of the art methods rely on reinforcement learning algorithms, where the actions to be taken by the robot in order to perform a manipulation task are learned from a large number of attempts of solving that task and, in a similar way to how dogs are trained, every time the robot manages to solve the task, the robot is given a reward. The main goal of the robot under this logic is to obtain the maximum reward possible.

Despite the great performance of reinforcement learning algorithms, when the task to be learned is very complex, these methods require a prohibitively large number of interactions with the environment, meaning that the use of these algorithms becomes infeasible due to the huge training time that they require unless very specific hardware is available.

To address this, Francisco has investigated alternative solutions  during his PhD. He first investigated  a novel reward that encourages a cube-lifting behaviour and reduces,  with respect to the previous state-of-the-art method, the number of interactions needed to learn how to manipulate a cube to follow randomly set trajectories with a three-fingered robot. Secondly, a novel framework to perform in-hand arbitrary 3D rotations with a cube was proposed based on a sequential execution of primitives that are capable of performing rotations around the axes, which obtains similar performance to current state of the art methods while requiring 37 million less interactions with the environment. Finally he introduced novel method, named QMP-HER, that makes automatic use of primitives that are simple to learn in order to improve the performance and the learning pace of current state of the art methods in several different complex cube manipulation tasks.