Work inspired by Piaget’s ideas tested a machine learning model that simulates the process of sensorimotor development in babies. The objective is to create a robot capable of improving its functioning on its own as it is exposed to new experiences.
by Malena Stariolo, translated by Leonardo Rossi
A master’s thesis, defended in the postgraduate program in Electrical Engineering at the Unesp Institute of Science and Technology in Sorocaba, won first place in the 2022 Robotics Thesis and Dissertation Competition, held by the Brazilian Computing Society (SBC) . The announcement was made on October 19, during Robotics 2022, the largest event in the area in Latin America, which brings together research, projects and competitions in robotics and artificial intelligence (AI). The research, conducted by Leonardo de Lellis Rossi and guided by Alexandre da Silva Simões, sought to simulate, in a robot, the learning stages experienced by a human baby in its initial months of life, in order to test a new model of learning machine that depends on the experiences and memories of the robot itself.
Rossi says that the interest in the research topic arose due to the “science fiction” air that the theme evokes. In two chapters of the dissertation, he used as epigraphs phrases taken from two classic works that deal with AI, the film Matrix and the book “I, Robot”. Equally important were the previous studies on the subject of machine consciousness led by Alexandre Simões, and Rossi continues to deepen the investigation, now in his doctorate.
Simões, who is a professor at the Unesp Institute of Science and Technology, says that for years the field of robotics has been looking for ways to develop robots that are endowed with such a degree of autonomy that allows them to adapt to situations that deviate from the scenarios foreseen in their initial programming. The objective is for machines to be able to deal with unforeseen events and choose the best way to act in the face of them.
This approach began when the research community realized that, before the advent of machine learning, the development of AI led to very limited robots that were only effective in very specific and controlled situations. In these very controlled scenarios, it is possible to describe all the variables and “unforeseen events” that the robot will encounter and determine how it should react at each moment. However, when something deviates even slightly from expectations, the machine is unable to respond to the event.
Trying to solve this issue, machine learning techniques have gained more and more visibility. “It’s much easier to let any agent learn, whether it’s software or a robot, than trying to explain everything to them,” says Simões.
Learn like a human being
One of the strands of study, in the field of machine learning, is called cognitive systems. It involves the development of artificial intelligence systems inspired by the human cognitive system. Researchers in this line are looking for ways to make robots capable of coordinating a series of different processes, such as memory, reasoning, planning and learning, so that the agent can have more complex reactions.
One of the hypotheses underlying this line of research is that, easier than trying to set up a system with knowledge equivalent to that of an adult human, it would be to allow the machine to acquire these skills gradually, going through stages similar to those of childhood. This approach allows the initial system to be technically much simpler and for the machine itself to improve it, as it accumulates experience.
“The idea is that he has elementary programming and, over time, he formulates his own memories, erases other memories, combines concepts and, in this way, he learns, based on the connections he makes with previous experiences”, explains Simões. The big question in the field is how to make robots capable of creating these connections, and linking knowledge, based on what they learn and experience.
Rossi’s dissertation followed this line and sought to develop initial models that would enable machine learning via cognitive systems. Entitled “Sensory-motor learning in cognitive robots using the CST-CONAIM model”, the work was also co-supervised by professor Esther Luna Colombini, from Unicamp. The objective was to carry out experiments on a robot that simulated the initial months of a baby’s life, to verify whether the model, developed by the group of which it is part, was capable of making the robot perform the expected learning actions. The young researcher sought, in Jean Piaget’s child psychology, the theoretical bases for the development of the model. The Swiss researcher is globally recognized for his theory called “genetic epistemology”. According to her, human beings are subjected to stages of cognitive development, from the moment of their birth. Piaget described the characteristics of each stage and substage, the first of which is sensorimotor. Based on this theoretical construction, Rossi carried out three experiments to test the model and simulate initial sensorimotor learning, described in the first three substages of Piaget’s theory, also known as the use of reflexes, primary circular reactions and secondary circular reactions. “Before we started walking, we had to first explore and get to know our own body, understand our motor capacity and our intention with the environment”, writes Rossi in the work.
According to Piaget’s theory, in the first sensorimotor substage, which occurs between birth and the first month of life, the baby increasingly demonstrates more control over his reflexes and begins to engage in standard behaviors, such as breastfeeding or holding. someone else’s finger. Then, in the second substage, between the 1st and 4th month, the baby starts to repeat actions that arouse pleasant sensations, on purpose. At this point, intentionality comes into existence. Finally, the third substage occurs between the 4th and 8th months of life and is related to anticipation. Over this time, babies have an increase in the coordination of their actions and also begin to anticipate what will happen, in order to direct their focus. “He is able to understand that the ball is going to land in a certain place, so he is already looking there, to confirm that this is going to happen”, explains Simões.
Putting theory into practice
For the simulation, Rossi used the robot Marta, a member of the research group and named after Marta da Silva, a football player. The experiments aimed to explore the ability to learn how to track and track another mobile agent. To do this, the robot was equipped with a camera at the height of its head, to simulate vision, and was positioned in an environment with blue and green cubes, of varying sizes, and with a mobile robot, the Pioneer, which moved around the space and dodge the cubes.
In the first experiment, which sought to simulate the sub-stage related to reflexes, the robot Marta was placed in the test area with the model in the initial stage, responsible only for making her perceive the existence of the objects that surrounded her. “At first, she only responded reactively to the stimuli she received from the environment. She didn’t show curiosity, motivation or intentionality,” comments Rossi. The process aimed to identify whether she was capable of developing reflexes. To do this, the researchers observed the robot’s attention span when the Pioneer passed close to it, in addition to the movement of the field of vision.
The procedure was restarted every time the agent fell, or every 5 consecutive interactions in which she did not show that she was paying attention to the other agent, even if he entered her field of vision. With each action, Marta received positive or negative stimuli, represented by scores, depending on her ability to maintain attention or not. At the end of the first experiment, the robot was already able to direct its field of vision and maintain focus, in order to increase its reward.
In the second experiment, Marta developed curiosity and motivation based on previously acquired reflex reactions. Its motivation function made her feel the need to acquire more knowledge. Here, the robot decided to focus and repeat actions that it realized attracted greater rewards. This included, for example, keeping her attention on the mobile robot when it was close to her body. It is important to highlight that in the first two sub-stages Marta’s actions were only related to her pre-programming and external stimuli. Rossi says that this pre-programmed character can be considered as the equivalent of human instincts.
Finally, in the last experiment Marta developed intentionality, that is, she stopped responding just instinctively and started actively directing her attention to things that were interesting to her. Marta kept her full attention on the Pioneer robot, taking advantage of the experiences acquired in the other two stages and realizing that, in this way, her rewards were maximized. This way, even when he moved away, the robot remained attentive, observing her path.
The experiments demonstrated that the robot Marta was able to gradually increase its capacity for attention and interaction with objects in the scenario, reusing and improving actions that were positively rewarded. At the end of the tests, the robot managed to develop the ability to track objects and show more interest in moving objects than in static ones.
Simões recognizes that the area of cognitive systems is still in its initial stages, and that research faces difficulties intrinsic to this current stage. “When someone works in regions that are at the frontier of knowledge, knowledge is still very dispersed. For example, the language that one author uses to define a topic is not the same as that used by another author, even if they are talking about the same thing,” he says. Despite the difficulties, Simões argues that cognitive systems will be the next wave of studies in artificial intelligence.
In this sense, on November 7, 2021, the Artificial Intelligence and Cognitive Architectures Hub (HIAAC) was created at Unicamp. The initiative aims to bring together researchers to develop studies aimed at expanding the cognitive and decision-making capabilities of mobile devices. Professor Simões is part of the hub team, in the Learning and Cognitive Theories line. Rossi is also part of the group, holding a PhD scholarship in the area of Cognitive Architectures. Currently, Rossi is developing his doctorate in the same line, at the Faculty of Electrical and Computer Engineering, at Unicamp, under the guidance of Professor Ricardo Ribeiro Gudwin and co-supervision of Professor Colombini, also members of the hub.
“Another point that motivates me in this area is its multidisciplinary nature and the discussions that occur, both in our engineering, computing and exact contexts, as well as in human, philosophical and psychological contexts. The human mind is incredible, it has been studied in different ways, but we still wonder about how it works”, says Rossi.