SIGEVOlution

Background

To realise the vision outlined above, any evolutionary process must be able to simultaneously evolve both the body and brain: while a naïve approach might simply consider adapting a brain to a newly evolved body design, it has been clear for some years that intelligence is not simply a property of the brain but also of the body. In their seminal text of 2006, Pfiefer and Bongard clearly argued that intelligence is at the same time both tightly constrained and also enabled by the body, and influenced by its morphology and its material properties. Hence any evolutionary process must jointly consider both options in order to produce an effective design. There are large potential payoffs in such an approach – devolving some intelligence to the morphology can reduce the need for complexity in the brain, with evolution offering the ideal tool to determine where along this spectrum the optimal balance lies by simultaneously evolving body and brain.

This challenge has been readily embraced by the evolutionary robotics community. In 2000, Lipson and Pollack published the first work that showed that small robots constructed from bars connected with free joints could be evolved in simulation and then built-in hardware post-evolution. A decade later, the same group later demonstrated that it was possible to apply the same principles in a vastly expanded design space that exploited a continuous distribution of hard and soft materials, where actuation was achieved via volumetric expansion or contraction of the materials themselves. In 2020, a very similar approach by Kriegman et al. resulted in the ground-breaking demonstration of xenobots: robots evolved in simulation and manufactured post-simulation using living stem-cells. However, while each of these examples represent noteworthy landmarks in the field, they all have two shortcomings. Firstly, none of the robots just described have sensors, hence although capable of directed motion, the lack of an ability to sense their environment results in an open-loop form of control that prevents the use of feedback for self-correction of behaviour. Secondly, fabricating post-evolution introduces a reality gap: infamous in robotics, this results from inevitable differences between simulation and reality, regardless of the fidelity of the simulator.

This can be avoided by bypassing the simulation stage altogether and evaluating evolved designs directly in hardware. This vision was first outlined in 2012, and has been greatly facilitated by advances in 3d printing and automation that facilitate rapid fabrication and autonomous assembly. In 2015, Brodbeck et al. demonstrated that robot evolution could be achieved completely in hardware: modular designs evolved in a computer built by a ‘mother robot’ directly in hardware, which then observes the behaviour of each new robot using a camera system in order to assign a fitness score. An alternative approach was proposed by Eiben et al. 2016, in which physical robots co-exist and interact in a real-world environment. The evolutionary process is decentralised: reproduction operators act on the genomes of robots that come within a pre-defined distance of each other and trigger the production of offspring.

ARE builds on this previous research, using a centralised evolutionary process and fully automated reproduction via a bespoke design of a facility called the RoboFab (figure 2). Furthermore, the design space is considerably enriched: moving away from modular designs with an algorithm that can design a robot skeleton of any shape printed using 3d plastic, and a varied array of sensor and actuators available, including both limbs and joints. There are considerable engineering challenges associated with designing a fully functioning RoboFab: these are discussed in detail in other articles: here, we provide an overview of the aspects associated with the design and implementation of the evolutionary mechanisms.

Evolution

The EvoSphere evolves robots using a genome that specifies both body and brain, where brains are neural controllers. Separate encodings are used for each part. Bodies are represented using a compositional pattern-producing network (CPPN): this is an indirect encoding commonly used in robotics as it is capable of generating repeating patterns, hence useful in evolving symmetry. An indirect encoding is also necessary for the brain: a neural controller cannot be directly specified on the genome as the number of inputs and outputs have to match the body plan generated by the CPPN in the body part of the genome. Hence, a second CPPN is encoded on the genome to generate the weights of a controller with an appropriate structure (see figure 3).

However, this also challenges the evolutionary process in that while reproduction between two morphologically distinct parents might result in a viable body plan, a directly inherited controller is at best unlikely to provide adequate control. One way to address this is to add a learning cycle into the evolutionary loop. This can either improve an inherited controller over an individual’s lifetime – when the inherited controller has an appropriate structure – or even learn a new controller from scratch. We propose an additional mechanism that can bootstrap the learning phase from a repository of previously discovered high-performing controllers: good controllers discovered in the evolutionary process are stored in an external repository, sorted by their `species’, where a species is defined by a simple descriptor denoting (e.g.) the number of wheels, joints, and sensors of each type on a robot. Whenever a new body is created, if there is already a controller in the repository for that species, then the learning process is bootstrapped using the controller from the repository, which has been shown to considerably speed up learning. (Note however that there can be many robots matching the species descriptor as it does not account for the skeleton shape or positioning of any attached components). The repository is continually updated as more and more species are discovered.

As shown in figure 4, the system thus contains nested evolutionary and learning algorithms. Specifically, we use NEAT to evolve the CPPN for the body-part, and a novelty-driven increasing population evolutionary strategy (NIP-ES) to implement the learning algorithm.

ECOSYSTEM Manager

Evolution in hardware is slow, mainly due to the time associated with printing and assembling the robot. Hence, in addition to running evolution in hardware, we simultaneously run a simulated version of the EvoSphere – this can be used to quickly explore the potential space of useful robots, identifying useful designs to be added to the physically evolving population. At the same time, a more exploitative EA is running in a much smaller physical population, mitigating the infamous reality gap. Robots can be transferred directly from virtual to physical worlds – or even vice versa. In addition, this also facilitates a novel form of reproduction in which offspring can be produced from a virtual mother and physical father (see figure 5). A human operator can control the entire process – managing the rate of migration of robots from one system to the other, acting as a ‘breeder’ in selecting robots for reproduction, or altering the parameters of the evolutionary algorithms. Examples of some of the robots evolved are shown in figure 6.

Looking Ahead

The original motivation for this work stemmed from a vision to create a system that could design robots in-situ in hardware – so that they can be fit for their desired purpose and avoid the infamous reality gap. However, from another perspective, in the words of the renowned biologist John Maynard-Smith 'until we discover extra-terrestrial life, biologists have only one “system” on which to study evolution’. Just as the Large Hadron Collider provides us with an instrument to study the intricacies of particle physics, perhaps a reproducing system of robots provides a new instrument to study fundamental questions about life itself. The ARE technology could be used for exactly such a purpose, providing a platform by which to generalise about evolution while simultaneously providing a springboard to explore a rich set of “what-if” questions, freed from the constraints imposed by working with biological matter.

Further resources

• TED Talk (from TEDWomen 2021): Self-assembling robots and the potential of artificial evolution

• New Scientist Article (March 2022): Meet the robots that can reproduce, learn and evolve all by themselves

Acknowledgements. The ARE project is funded by EPSRC (EP/R03561X, EP/R035733, EP/R035679) and is a collaboration between the University of York (Prof. A. Tyrrell), Edinburgh Napier University (Prof. E. Hart), Bristol Robotics Laboratory (Prof. A. Winfield), University of Sunderland (Prof. J. Timmis), VU Amsterdam (Prof. A.E. Eiben). The vast majority of the work is carried out by Dr Léni Le Goff; Dr Edgar Buchanan; Dr Wei Li; Dr Matt Hale; Mike Angus; Rob Woolley; Matteo De Carlo.

Forthcoming Events