GenesisCube

Evolutionary Algorithm Explained

Evolutionary algorithms will see increased use and development due to the increased availability of computation, more robust, and the increasing demand for other artificial intelligence techniques.
Evolutionary Computation approaches can offer a reliable and effective approach to address complex problems in real-world applications.
Evolutionary Computation algorithms have recently been used to improve the performance of Machine Learning models. Evolutionary approaches can be used in preprocessing, learning and postprocessing (e.g., hyperparameter optimization). This post investigates the role of evolutionary algorithms in solving machine learning challenges.

What is Evolutionary Algorithm?

The most powerful machine learning techniques are inspired by nature. The terminology of natural computation has two essential sources: taking inspiration from nature and employing computers. Evolutionary Computation (EC) approaches power stemming from the process’s nature used to produce intelligent organisms.
The processes applied in EC are inspired by natural evolution, they can be expected to be efficient and effective. EC algorithms generally work with populations of individuals that are associated with a specific problem to be solved.

While evolution and learning are two aspects of adaptation artificial systems, one can discern them on the basis of the lifetime of an individual. We speak of learning adaptation if an individual, during its lifetime, adapts to a certain problem domain. Where evolution and learning complement each other, there is a bidirectional relationship in computing between EC techniques and learning algorithms so they can be combined to attack complex optimization problems in different domains together.

In simple words, learning algorithms are integrated into evolutionary techniques to address problems with evolutionary computation approaches, such as being trapped in local optima and premature convergence. Before give to evolutionary algorithms, lets us learn more about some concepts of evolutionary algorithm.

Population

Population is a set of solution candidates or entities. Each population is created temporally and has the potential to age with each evolutionary cycle i.e. generation. The initial population is either seeded randomly or is sampled from a set of known solutions.
Entities can die or thrive between generations. The size of the population can be fixed or dynamic throughout the evolutionary process. Fixed is when the number of entities is kept constant, and dynamic is when the population size changes. If dynamic, a larger initial population may bring some potential advantages. This is especially true when choosing the first strong candidates for further evolution.

Population Entities

The population entities also known as phenotypes, can be any type provided that type allows recombination and mutation to be applied. The main styles are strings and programs. In this context programs are a loose term. The strings are fixed length and more likely represent variables or parameters. Effectively any structure from a traditional computer programming language to hardware layout, or even biological descriptions, can be a program.

Generation

Generation is a specific step in an evolutionary population, where individuals in the population undergo change via crossover and mutation. For evolutionary algorithms with a fixed-run a restriction is imposed on the number of generations. By contrast, continuous evolutionary algorithms have no upper bounds. Evolutionary algorithms can vary from small populations with high number of generations to large populations with significantly lower number of generations. These trade-offs need to be made in terms of computation time.

For example, some scenarios will have expensive evaluation functions so fewer generations with larger populations might be preferred. The number of generations required is both algorithmic and problem-domain specific. The children in the next generation can also include the parents from the previous generation. This is called elitism, where the strongest entities remain in the population.

Differential Evolution Algorithm

Differential evolution is a type of evolutionary algorithm useful mainly for the function optimization in continuous search space. The main advantages of differential evolution are: It is easy to use, and it has efficient memory utilization, a lower computational effort, and it scales better when handling large problems. The main idea of the differential evolution algorithm is connected with computing the difference between two individuals chosen randomly from the population. Differential evolution determines the function gradient within a given area not at a single point. Therefore, differential evolution algorithm prevents the solution of sticking at a local extreme of the optimized function.

Evolution Strategies

The evolution strategies are different, mainly in the selection procedure. In the evolution strategies, a temporary population is created; it has the different size than the parental population. In this step, the fitness values are not important. Individuals in the temporary population undergo crossover and mutations. From such populations, an assumed number of the best individuals are selected to the next generation of the population. Evolution strategies operate on the vectors of the floating-point numbers.

Evolutionary Programming

Evolutionary programming was developed as a tool for discovering the grammar of the unknown language. However, evolutionary programming became more popular when it was proposed as the numerical optimization technique. Evolutionary programming is similar to the evolution strategies, but with one difference. In evolutionary programming, the new population of individuals is created by mutating every individual from the parental population, while in the evolutionary strategies, every individual has the same probability to be selected to the temporary population on which the genetic operations are performed.

In evolutionary programming, the mutation is based on the random perturbation of the values of the particular genes of the mutated individual. The newly created and the parental populations are the same sizes. Finally, the new generation of the population is created using the ranking selection of the individuals from both, the parental and the mutated populations.

Genetic Algorithms

Genetic Algorithms are the most common among the evolutionary algorithms. Genetic Algorithm applies evolution to fixed length strings. The length of the string represents the dimensionality of the problem. These strings represent variables or parameters and are useful when exploring a large number problem-domain space.
This space is normally beyond human or traditional methods. Variables or parameters are converted to fixed length strings, the strings are entities in the population and are evolved using crossover windows and mutation. Genetic Algorithms handle optimization and configuration problems where there are too many variables or parameters for a traditional method to succeed. The variables or parameters may interact making a potential solution much harder to identify.

Genetic Programming

Genetic Programming manipulate structures and in-particular executable programs or mathematical equations. Early Genetic Programmings were based on tree representations and used the LISP programming language as the grammar. More recently there have been other representations introduced and newer languages such as Python have become popular as the main target.

Use cases of Evolutionary Algorithms

Evolutionary Algorithms are suitable mainly for optimization, scheduling, planning, and management problems. If we analyze, we can see that the EAs methods are mainly used in the following areas.

• Engineering electrical electronics
• Computer science theory methods
• Computer science interdisciplinary applications
• Automation control system
• Computer science information systems
• Operations research management science

Recommended for you:

• Sarsa Algorithm Reinforcement Learning
• Genetic Algorithm in Machine Learning