SIGEVOlution

Tributes to Julian F. Miller (1955 – 2022)

Wolfgang Banzhaf, James A. Foster, Simon Harding, Roman Kalkreuth, Gul Muhammad Khan, W.B. Langdon, Riccardo Poli, Stephen Smith, Susan Stepney, Martin Trefzer and Dennis G. Wilson

Julian’s love for physical reality led him to develop what he called in materio evolution, where one evolves a physical substrate directly. He and his student, Simon Harding, used liquid crystal (LC) displays as substrates for evolving circuits. They attached leads for electrical input and output, then evolved LC configurations directly, selecting for input-output mappings that implemented signal processing circuits. Simon built one of these systems for my own lab when he did a postdoc with me. I enjoyed long nights brainstorming with Julian about other physical systems one might evolve, from steam pipes to quantum dots, as I recall.

Julian was a political activist when he had to be. Once when I was staying with him in York, he told me that the field behind his house was the site of an ancient battle. It was slated for development. Julian led a movement to preserve the archaeology instead.

Julian enjoyed abstraction but was always happier with the tangible. This was one reason he took up sculpting. He showed me pictures of some of his sculptures. Ironically, I never saw one in person—only in the form of digital images. They were abstract, but bursting with emotions like longing and joy. Emotion made physical. Like Julian.

Julian also wrote poetry, in a style uniquely his. Here is a couplet from a sonnet of his titled “You are made of…”. I find the invented words perfectly expressive.

"You have seen the coolmindlight and mathrhyme.

Most here are glumfoolish and sparklenought."

I will miss Julian: scientist, engineer, thinker, artist, friend.

We worked together on many projects over the years, but I think the “evolution in-materio” was the highlight and most profound. His concept was that the physical properties of the universe could be usefully unlocked using the simplest of algorithms. He marvelled at how all life was created using the same process, and this gave him a deep appreciation for the world around him – which also inspired his artistic interests.

Having toured conferences with Julian many times I was always amazed by how everyone knew him, everyone liked him and everyone wanted to hear what he had to talk about. Julian loved to talk and share his ideas and passions. I remember a joint two hour session, where my 30 minutes was squeezed down to a few minutes in extra time as Julian had got carried away with the excitement of his part of the presentation. I remember even though we continued through the break, the whole audience stayed to hear what Julian had to say.

Never one for the mainstream or group-think, Julian wanted to work on concepts that should change the world – one unconventional idea after another. They certainly changed mine.

I will never forget our meeting in a small cafe in the lovely city of Salisbury (England, UK) back in 2016 where we had a great discussion about recent developments in Genetic Programming (GP) while having good tea and cake. At that time, I was on the verge of proposing a new idea for graph recombination and was slightly nervous since recombination was considered a big unresolved issue in CGP.

Despite all earlier failures of recombination in CGP, Julian encouraged the follow-up of its adaption. That impressed me about Julian. He never lost his scientific vision and passion. Julian officially retired at the University of York in 2016 but followed up his scientific life until his death in February 2022. After his retirement he concentrated on the so-called CGP Brain Project which has a visionary character:

"For a few years now I have been working on using CGP to evolve two programs that develop a whole neural network and can learn to solve multiple problems simultaneously. One evolved program controls the body of a neuron and the other controls the dendrites. When you run the programs it builds an entire brain-like network which can solve three problems simultaneously" – Julian Miller (Statement on www.cartesiangp.com)

Julian has been a huge inspiration throughout my academic life and for my research in CGP. Overall, CGP research has made great progress over the last two decades and I rejoice in Julian's passionate scientific effort. The evolutionary computation community has lost a great personality.

Thank you, Julian! You will be greatly missed.

He learned from nature, the way it organized itself and survived over the years, he said all the secrets lie inside it. I had a chance to work with him, we built neural developmental models inspired by the biological brain and explored its abilities in various game scenarios. We also worked on implementation of methodologies for automatic design of Artificial Neural networks, both feedback and feed forward and contributed in the field of neuro-evolution. His understanding was that if something doesn’t change, it cannot learn, thus plastic Neural Networks concepts were born, and exploited for generalization and robustness. He never stops learning, experimenting and contributing to science, he was a true scientist complete in all aspects.

Over time we got attached socially, and I got to know about his family. He was a wonderful son for choosing to stay single for 18 years, to dedicate his time to his unwell mother. He talked about his father and his art, he was proud to be a son to a world war hero. He was a caring brother; he was happy to have met his brother after so long. If it wasn’t for his cancer, he would have continued to contribute to science. We even had a discussion on this new model for neural development, just a few weeks before he got hospitalized. We had plans of continuing to work together. I pray he rests in peace.

If all Julian had contributed to science was the now-established field of CGP, that alone would be a worthwhile legacy. But he also contributed the now-burgeoning field of in materio computing: of computing directly in physical substrates, such as liquid crystals and carbon nanotubes. And in his "retirement" years, he was working a new evolved developmental brain model, and publishing right up to the end. Who knows what else he would have done, had he not been so cruelly struck down.

And, of course, who could forget the acronyms? Projects and proposals delightfully dubbed

SEEDS, NASCENCE (and the follow up, RECONASENSE), IMPROBED, and more:

Julian had a way with letters.

I will miss him.

Julian had a hands-on mentality and he explored initial ideas on in-materio computing using a liquid-crystal display that his PhD student rescued from a pile of electronic recycling. With this maverick setup, they made significant discoveries on the influence of the test setup on the outcome of experiments.

I remember many philosophical discussions on diverse topics over lunch. A particular one that comes to mind is when we talked about evolution, extraterrestrials and the film K-PAX. The film is about PROT who claims to be from a faraway planet and changes the way his therapist Dr. Powell, and the audience, looks at the world:

(Prot) "You know what I've learned about your planet? There's enough life on Earth to fill 50 planets. Plants, animals, people, fungi, viruses, all jostling to find their place, bouncing off each other, feeding off each other. Connected."

(Dr Powell) "You don't have that kind of connection on K-PAX?"

(Prot) "Nobody wants, nobody needs. On K-PAX, when I'm gone, nobody misses me. There would be no reason to. And yet I sense that when I leave here -- I will be missed. Yes. Strange feeling.”

Julian had a very clear notion of what was important and interesting for him to study and he was fundamental in advancing those topics. He was excited to see other colleagues advance on the topic of neural development and aimed to cultivate discussion and collaboration on this topic. He was always eager to learn from others, especially biologists whose insights into neural functionality inspired him. He was an example of how a senior researcher, with an established field developed and led by him, could be inspired by a new topic, asking questions and integrating new insights into his model. He was intensely curious about the brain and pursued his research on its intelligence until his last moments. Research in development and structural learning in neural networks is advancing and Julian has been recognized as a pioneer in this domain, well ahead of his time. While that time has now ended, I believe that Julian's ideas will resonate and inspire others for many years to come, as they have inspired me. Farewell, dear friend, and thank you.

Evolving Open Complexity

W.B. Langdon, University College London, London, UK

Abstract

Information-theoretic analysis of large evolved programs produced by running genetic programming for up to a million generations has shown even functions as smooth and well behaved as floating-point addition and multiplication lose entropy and consequently are robust and fail to propagate disruption to their outputs. This means that, while dependent upon fitness tests, many genetic changes deep within trees are silent. For evolution to proceed at a reasonable rate it must be possible to measure the impact of most code changes, yet in large trees, most crossover sites are distant from the root node. We suggest that to evolve very large, very complex programs, it will be necessary to adopt an open architecture where most mutation sites are within 10–100 levels of the organism's environment.

Background

Recently we have been investigating the long-term evolution of genetic programming. Firstly, using Poli’s submachine code GP [24, 25], evolving large binary Boolean trees [7] and more recently exploiting SIMD Intel AVX and multi-core parallelism to evolve floating-point GP [9, 14, 12, 5]. Running for up to a million generations without size limits has generated, at more than a billion nodes, the biggest programs yet evolved and forced the development [10, 11] of, at the equivalent of more than a trillion GP operations per second, the fastest GP system. It has also prompted information- theoretic analysis of programs [16]. (Of course, information theory has long been used with evolutionary computing, e.g. [2].)

One immediately applicable result has been the realisation that in deep GP trees most changes have no impact on fitness and once this has been proved, for a given child, its fitness evaluation can be cut short, and fitness simply copied from the parent. This can lead to enormous speedups [13].

We have also considered traditional imperative (human-written) programs and shown these too are much more robust than is often assumed [15, 6, 8]. Indeed, we suggest that information theory provides a unified view of the difficulty of testing software [17, 1, 22].

The question of why fitness is so often exactly inherited [20, 21, 23] despite brutal genetic change is answered by the realisation that without side effects the disruption caused by the mutation must be propagated up the tree through a long chain of irreversible operations to the root node. Each function in the chain can lose entropy. In many cases, deeply nested functions progressively lose information about the perturbation as the disruption fails to propagate to the program’s output. Thus, the mutation becomes invisible to fitness testing and its utility cannot be measured. Without fitness selective pressure, evolution degenerates into an undirected random walk.

In bloated structures, information loss leads, from an evolutionary point of view, to extremely high resilience, robustness and so stasis. From the engineering point of view, this is problematic, as then almost all genetic changes have no impact and evolutionary progress slows to a dawdle.

Since all digital computing is irreversible [18] it inherently loses information and so without shortcuts, must lead to failed disruption propagation (FDP) [22]. We suggest in order to evolve large complex systems it must be possible to measure the impact of genetic changes, therefore we must control FDP and suggest in the next section that to evolve large systems, they be composed of many programs of limited nesting depth and structured to allow rapid communication of both inputs and outputs to the (fitness determining) environment.

Open Complex System

Can we evolve systems more like cell interiors, with high surface area membranes composed of very many small adjacent programs each of limited depth placed side by side? The membranes form an open structure with many gaps between them. The gaps themselves support rapid communication (which might be implemented using global memory or enhanced communication links or buses) with no, or little, processing ability and consequently little information loss.

Figure 1 shows 1300 programs arranged in an open structure (based on fit Sextic polynomial [4] trees with average height 9.22, quartile range 7–11).

Rich Lenski, in his long-term evolution experiment (LTEE) [19, 3] has demonstrated that Nature can continue to innovate in a static environment for more than 75000 generations. We have shown GP can do similarly for at least 100000 generations. However, when evolving deep structures, progress slows dramatically and therefore we feel monolithic deep structures will not be sufficient to automatically evolve complex systems. Instead, an open structure like Figure 1 may be needed.

References

Call for Submissions

Parallel Problem Solving from Nature (PPSN 2022)

The 17th International Conference on Parallel Problem Solving from Nature (PPSN 2022) will be held in Dortmund (Germany), 10–14 September 2022. The workshops, tutorials and the conference will be held “on-site” only. The participants should take this into account in their planning. The local organizers will provide adequate rooms and technical equipment for local presentations. A hybrid format is not planned. In case of pandemic-caused restrictions, the event will have to switch to a fully online format.

PPSN was originally designed to bring together researchers and practitioners in the field of Natural Computing, the study of computing approaches that are gleaned from natural models. Today, the conference series has evolved and welcomes works on all types of iterative optimization heuristics. Notably, we also welcome submissions on connections between search heuristics and machine learning or other artificial intelligence approaches. Submissions covering the entire spectrum of work, ranging from rigorously derived mathematical results to carefully crafted empirical studies, are invited.

Important Dates

• Paper submission (extension): 13 April 2022

• Author notification: 6 June 2022

• Camera-ready: 25 June 2022

• Tutorials/Workshops: 10–11 September 2022

• Conference: 12–14 September 2022

Program Chairs: Hernan Aguirre, Pascal Kerschke, Gabriela Ochoa, Tea Tusar

Entries are hereby solicited for awards totalling $10,000 for human-competitive results that have been produced by any form of genetic and evolutionary computation (including, but not limited to genetic algorithms, genetic programming, evolution strategies, evolutionary programming, learning classifier systems, genetic improvement, grammatical evolution, gene expression programming, differential evolution, etc.) and that have been published in the open literature between the deadline for the previous competition and the deadline for the current competition.

Important Dates

• Friday, May 27, 2022 — Deadline for entries (consisting of one TEXT file, PDF files for one or more papers, and possible "in press" documentation (explained below). Please send entries to goodman at msu dot edu.

• Friday, June 10, 2022 — Finalists will be notified by e-mail.

• Friday, June 24, 2022 — Finalists not presenting in person must submit a 10-minute video presentation (or the link and instructions for downloading the presentation, NOT a YouTube link) to goodman at msu dot edu.

• July 9-13, 2022 (Saturday - Wednesday) — GECCO conference (the schedule for the Humies session is not yet final, so please check the GECCO program as it is updated).

• Monday, July 11, 2022 Presentation session.

• Wednesday, July 13, 2023 — Announcement of awards at the plenary session of the GECCO conference.

Judging Committee: Erik Goodman, Una-May O'Reilly, Wolfgang Banzhaf, Darrell Whitley, Lee Spector, Stephanie Forrest

Publicity Chair: William Langdon