Large language models help programs to evolve

Abstract

Usingthis representation, a genetic-programming system 'mutates' a program by randomly changing one node in the tree to a different value (Fig. la). Instead of replacing random parts of a syntax tree, an LLM can generate a variation of a program written in a standard programming language, such as Python. To do so, a simple, but powerful, approach is to select two programs, concatenate them, and ask the LLM to complete the program using the concatenated pair as a prompt - resulting in the generation of a third program (Fig. 1b). Romera-Paredes et al. used this fresh approach to genetic programming to find ways of solving mathematical problems in optimization and geometry that were better than the best attempts of human programmers.