The Bayesian Optimization Algorithm with Substructural Local Search

Sep 18, 20075 likes1,158 views

This work studies the utility of using substructural neighborhoods for local search in the Bayesian optimization algorithm (BOA). The probabilistic model of BOA, which automatically identifies important problem substructures, is used to define the structure of the neighborhoods used in local search. Additionally, a surrogate fitness model is considered to evaluate the improvement of the local search steps. The results show that performing substructural local search in BOA significatively reduces the number of generations necessary to converge to optimal solutions and thus provides substantial speedups.

The Bayesian Optimization Algorithm with Substructural Local Search

This document discusses information geometry and its applications in statistical signal processing. It introduces several key concepts: 1) Statistical signal processing models data with probability distributions like Gaussians and histograms. Information geometry provides a geometric framework for intuitive reasoning about these statistical models. 2) Exponential family mixture models generalize Gaussian and Rayleigh mixtures and are algorithmically useful in dually flat spaces. 3) Distances between statistical models, like Kullback-Leibler divergence and Bregman divergences, can be interpreted geometrically in terms of convex conjugates and Legendre transformations.

Iclr2016 vaeまとめDeep Learning JP

This document summarizes a presentation about variational autoencoders (VAEs) presented at the ICLR 2016 conference. The document discusses 5 VAE-related papers presented at ICLR 2016, including Importance Weighted Autoencoders, The Variational Fair Autoencoder, Generating Images from Captions with Attention, Variational Gaussian Process, and Variationally Auto-Encoded Deep Gaussian Processes. It also provides background on variational inference and VAEs, explaining how VAEs use neural networks to model probability distributions and maximize a lower bound on the log likelihood.

Inference & Learning in Linear Chain Conditional Random Fields (CRFs)Anmol Dwivedi

This mini-project will consider performing inference and learning in Linear Chain CRFs. In particular, it will consider an application to hand-written word recognition. Handwritten word recognition is a task many have explored with different methods of machine learning. Some written characters can be evaluated individually or as a whole word to account for the context in characters. In this mini-project, we use linear chain CRF models to account for context between the characters of a word to improve word recognition accuracy.

（DL輪読）Variational Dropout Sparsifies Deep Neural NetworksMasahiro Suzuki

Tutorial on Markov Random Fields (MRFs) for Computer Vision ApplicationsAnmol Dwivedi

Snakes in Images (Active contour tutorial)Yan Xu

The document describes how snakes, or active contours, can be used to model shapes in images. It discusses how snakes work by defining an energy function along a curve and minimizing that energy to find the optimal curve. The energy includes an internal term based on curvature and an external term from image features. Level sets are used to propagate the curves towards the minimum energy configuration using gradient descent. Key steps include modeling the shape as a curve, defining the energy function, deriving the curve to minimize energy via calculus of variations, and propagating the curves using level sets.

ALEXANDER FRACTIONAL INTEGRAL FILTERING OF WAVELET COEFFICIENTS FOR IMAGE DEN...sipij

The present paper, proposes an efficient denoising algorithm which works well for images corrupted with Gaussian and speckle noise. The denoising algorithm utilizes the alexander fractional integral filter which works by the construction of fractional masks window computed using alexander polynomial. Prior to the application of the designed filter, the corrupted image is decomposed using symlet wavelet from which only the horizontal, vertical and diagonal components are denoised using the alexander integral filter. Significant increase in the reconstruction quality was noticed when the approach was applied on the wavelet decomposed image rather than applying it directly on the noisy image. Quantitatively the results are evaluated using the peak signal to noise ratio (PSNR) which was 30.8059 on an average for images corrupted with Gaussian noise and 36.52 for images corrupted with speckle noise, which clearly outperforms the existing methods.

[DL輪読会]Generative Models of Visually Grounded ImaginationDeep Learning JP

The document proposes a new model for visually grounded semantic imagination that can generate images from linguistic descriptions of concepts specified by attributes. The model uses a variational autoencoder with three inference networks to handle images, attributes, and missing modalities. It represents the attribute inference distribution as the product of expert Gaussians, allowing generation of concepts not seen during training by combining learned attributes. The paper introduces three criteria for evaluating such models: correctness, coverage, and compositionality.

Image Restitution Using Non-Locally Centralized Sparse Representation ModelIJERA Editor

Sparse representation models uses a linear combination of a few atoms selected from an over-completed dictionary to code an image patch which have given good results in different image restitution applications. The reconstruction of the original image is not so accurate using traditional models of sparse representation to solve degradation problems which are blurring, noisy, and down-sampled. The goal of image restitution is to suppress the sparse coding noise and to improve the image quality by using the concept of sparse representation. To obtain a good sparse coding coefficients of the original image we exploit the image non-local self similarity and then by centralizing the sparse coding coefficients of the observation image to those estimates. This non-locally centralized sparse representation model outperforms standard sparse representation models in all aspects of image restitution problems including de-noising, de-blurring, and super-resolution.

(研究会輪読) Facial Landmark Detection by Deep Multi-task LearningMasahiro Suzuki

The document summarizes a research paper on facial landmark detection using deep multi-task learning. It proposes a Tasks-Constrained Deep Convolutional Network (TCDCN) that uses facial landmark detection as the main task and related auxiliary tasks like pose estimation and attribute inference to improve performance. The TCDCN learns shared representations across tasks using a deep convolutional network. It introduces task-wise early stopping to halt learning on auxiliary tasks that reach optimal performance early to avoid overfitting and improve convergence on the main task of landmark detection. Experimental results showed the proposed approach outperformed existing methods.

Chapter 2 Image Processing: Pixel RelationVarun Ojha

This document discusses pixel relationships and neighborhood concepts in digital images. It defines a pixel and pixel connectivity. There are different types of pixel neighborhoods, including 4-neighbor, 8-neighbor, and diagonal neighbors. Connected components are sets of pixels that are connected based on pixel adjacency. Algorithms can label connected components and identify distinct image regions. Various distance measures quantify how close pixels are, such as Euclidean, Manhattan, and chessboard distances. Arithmetic and logical operators can combine pixel values from different images. Neighborhood operations apply functions to pixels based on their values and those of nearby pixels.

Methods of Manifold Learning for Dimension Reduction of Large Data SetsRyan B Harvey, CSDP, CSM

Tailored Bregman Ball Trees for Effective Nearest NeighborsFrank Nielsen

This document presents an improved Bregman ball tree (BB-tree++) for efficient nearest neighbor search using Bregman divergences. The BB-tree++ speeds up construction using Bregman 2-means++ initialization and adapts the branching factor. It also handles symmetrized Bregman divergences and prioritizes closer nodes. Experiments on image retrieval with SIFT descriptors show the BB-tree++ outperforms the original BB-tree and random sampling, providing faster approximate nearest neighbor search.

Understanding Random Forests: From Theory to PracticeGilles Louppe

This document provides an overview of random forests and their implementation. It begins with motivating random forests as a way to reduce variance in decision trees. It then discusses growing and interpreting random forests through variable importances. The document presents theoretical results on the decomposition and properties of variable importances. It concludes by describing the efficient implementation of random forests in scikit-learn, including its modular design and optimizations for speed.

Continuous and Discrete-Time Analysis of SGDValentin De Bortoli

This document summarizes results on analyzing stochastic gradient descent (SGD) algorithms for minimizing convex functions. It shows that a continuous-time version of SGD (SGD-c) can strongly approximate the discrete-time version (SGD-d) under certain conditions. It also establishes that SGD achieves the minimax optimal convergence rate of O(t^-1/2) for α=1/2 by using an "averaging from the past" procedure, closing the gap between previous lower and upper bound results.

Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super VectorUnited States Air Force Academy

PMF BPMF and BPTFPei-Che Chang

(研究会輪読) Weight Uncertainty in Neural NetworksMasahiro Suzuki

Bayes by Backprop is a method for introducing weight uncertainty into neural networks using variational Bayesian learning. It represents each weight as a probability distribution rather than a fixed value. This allows the model to better assess uncertainty. The paper proposes Bayes by Backprop, which uses a simple approximate learning algorithm similar to backpropagation to learn the distributions over weights. Experiments show it achieves good results on classification, regression, and contextual bandit problems, outperforming standard regularization methods by capturing weight uncertainty.

Fuzzy Logic Analysis using GeoMedia by Bhaskar Reddy PulsaniMapWindow GIS

Site selection process is a screening technique, used to select appropriate sites for dumping waste. Screening is done by considering the restrictions that have to be met when selecting a site. Two Screening methodologies i.e. Boolean and Fuzzy were used for delineating dumping zones. Boolean defines a two valued logic with sharp delineation of boundaries where as fuzzy provides a smooth transition between the boundaries to handle the concept of vagueness. Implementation of membership functions for Fuzzy Logic analysis requires a lot of steps for manual process. As most of the process is generic for different layers, the analysis procedure was automated by customizing the application. Therefore, for site selection, a manual and automated fuzzy logic analysis was performed by making use of GeoMedia.

改进的固定点图像复原算法_英文_阎雪飞alen yan

This document summarizes an improved fixed point method for image restoration. It begins by introducing the problem of image restoration and describing common image degradation models. It then presents the traditional fixed point method using Tikhonov regularization. The improved method proposes changing the regularization parameter from larger to smaller values across iterations. Experimental results show the improved method performs better than other popular algorithms at solving motion and Gaussian degradation, producing reconstructed images with less noise and more detail.

(DL hacks輪読) How to Train Deep Variational Autoencoders and Probabilistic Lad...Masahiro Suzuki

This document discusses techniques for training deep variational autoencoders and probabilistic ladder networks. It proposes three advances: 1) Using an inference model similar to ladder networks with multiple stochastic layers, 2) Adding a warm-up period to keep units active early in training, and 3) Using batch normalization. These advances allow training models with up to five stochastic layers and achieve state-of-the-art log-likelihood results on benchmark datasets. The document explains variational autoencoders, probabilistic ladder networks, and how the proposed techniques parameterize the generative and inference models.

Macrocanonical models for texture synthesisValentin De Bortoli

This document discusses macrocanonical models for texture synthesis. It begins by introducing the goal of texture synthesis and providing a brief history. It then describes the parametric question of combining randomness and structure in images. Specifically, it discusses maximizing entropy under geometric constraints. The document goes on to discuss links to statistical physics, defining microcanonical and macrocanonical models. It focuses on studying the macrocanonical model, describing how to find optimal parameters through gradient descent and how to sample from the model using Langevin dynamics. The document provides examples of texture synthesis and compares results to other methods.

Gtti 10032021Valentin De Bortoli

1. The document presents Plug-and-Play priors for Bayesian imaging using Langevin-based sampling methods. 2. It introduces the Bayesian framework for image restoration and discusses challenges in modeling the prior. 3. A Plug-and-Play approach is proposed that uses an implicit prior defined by a denoising network in conjunction with Langevin sampling, termed PnP-ULA. Experiments demonstrate its effectiveness on image deblurring and inpainting tasks.

Iaetsd vlsi implementation of gabor filter based image edge detectionIaetsd Iaetsd

This document describes a VLSI implementation of an edge detection technique using Gabor filtering and rough clustering. The proposed technique smoothes images using Gabor filtering and performs edge detection using rough clustering. It was tested on various images and compared to other edge detection methods. The technique achieved noise-free and robust edge detection results. Finally, the technique was implemented in Verilog HDL and tested on a Xilinx FPGA for VLSI implementation.

Polynomial Matrix DecompositionsFörderverein Technische Fakultät

Polynomial matrices can help to elegantly formulate many broadband multi-sensor / multi-channel processing problems, and represent a direct extension of well-established narrowband techniques which typically involve eigen- (EVD) and singular value decompositions (SVD) for optimisation. Polynomial matrix decompositions extend the utility of the EVD to polynomial parahermitian matrices, and this talk presents a brief overview of such polynomial matrices, characteristics of the polynomial EVD (PEVD) and iterative algorithms for its solution. The presentation concludes with some surprising results when applying the PEVD to subband coding and broadband beamforming.

Reweighting and Boosting to uniforimty in HEParogozhnikov

This document discusses using machine learning boosting techniques to achieve uniformity in particle physics applications. It introduces the uBoost and uGB+FL (gradient boosting with flatness loss) approaches, which aim to produce flat predictions along features of interest, like particle mass. This provides advantages over standard boosting by reducing non-uniformities that could create false signals. The document also proposes a non-uniformity measure and minimizing this with a flatness loss term during gradient boosting training. Examples applying these techniques to rare decay analysis, particle identification, and triggering are shown to achieve more uniform efficiencies than standard boosting.

Maximum likelihood estimation of regularisation parameters in inverse problem...Valentin De Bortoli

This document discusses an empirical Bayesian approach for estimating regularization parameters in inverse problems using maximum likelihood estimation. It proposes the Stochastic Optimization with Unadjusted Langevin (SOUL) algorithm, which uses Markov chain sampling to approximate gradients in a stochastic projected gradient descent scheme for optimizing the regularization parameter. The algorithm is shown to converge to the maximum likelihood estimate under certain conditions on the log-likelihood and prior distributions.

Lec15 graph laplacian embeddingUnited States Air Force Academy

NIPS 2016. BayesOpt workshop invited talk.Joshua Knowles

An EducationJoshua Knowles

More Related Content

What's hot (20)

Image Restitution Using Non-Locally Centralized Sparse Representation ModelIJERA Editor

(研究会輪読) Facial Landmark Detection by Deep Multi-task LearningMasahiro Suzuki

Chapter 2 Image Processing: Pixel RelationVarun Ojha

Methods of Manifold Learning for Dimension Reduction of Large Data SetsRyan B Harvey, CSDP, CSM

Tailored Bregman Ball Trees for Effective Nearest NeighborsFrank Nielsen

Understanding Random Forests: From Theory to PracticeGilles Louppe

Continuous and Discrete-Time Analysis of SGDValentin De Bortoli

Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super VectorUnited States Air Force Academy

PMF BPMF and BPTFPei-Che Chang

(研究会輪読) Weight Uncertainty in Neural NetworksMasahiro Suzuki

Fuzzy Logic Analysis using GeoMedia by Bhaskar Reddy PulsaniMapWindow GIS

改进的固定点图像复原算法_英文_阎雪飞alen yan

(DL hacks輪読) How to Train Deep Variational Autoencoders and Probabilistic Lad...Masahiro Suzuki

Macrocanonical models for texture synthesisValentin De Bortoli

Gtti 10032021Valentin De Bortoli

Iaetsd vlsi implementation of gabor filter based image edge detectionIaetsd Iaetsd

Polynomial Matrix DecompositionsFörderverein Technische Fakultät

Reweighting and Boosting to uniforimty in HEParogozhnikov

Maximum likelihood estimation of regularisation parameters in inverse problem...Valentin De Bortoli

Lec15 graph laplacian embeddingUnited States Air Force Academy

Image Restitution Using Non-Locally Centralized Sparse Representation ModelIJERA Editor

(研究会輪読) Facial Landmark Detection by Deep Multi-task LearningMasahiro Suzuki

Chapter 2 Image Processing: Pixel RelationVarun Ojha

Methods of Manifold Learning for Dimension Reduction of Large Data SetsRyan B Harvey, CSDP, CSM

Tailored Bregman Ball Trees for Effective Nearest NeighborsFrank Nielsen

Understanding Random Forests: From Theory to PracticeGilles Louppe

Continuous and Discrete-Time Analysis of SGDValentin De Bortoli

Lec-08 Feature Aggregation II: Fisher Vector, AKULA and Super VectorUnited States Air Force Academy

PMF BPMF and BPTFPei-Che Chang

(研究会輪読) Weight Uncertainty in Neural NetworksMasahiro Suzuki

Fuzzy Logic Analysis using GeoMedia by Bhaskar Reddy PulsaniMapWindow GIS

改进的固定点图像复原算法_英文_阎雪飞alen yan

(DL hacks輪読) How to Train Deep Variational Autoencoders and Probabilistic Lad...Masahiro Suzuki

Macrocanonical models for texture synthesisValentin De Bortoli

Gtti 10032021Valentin De Bortoli

Iaetsd vlsi implementation of gabor filter based image edge detectionIaetsd Iaetsd

Polynomial Matrix DecompositionsFörderverein Technische Fakultät

Reweighting and Boosting to uniforimty in HEParogozhnikov

Maximum likelihood estimation of regularisation parameters in inverse problem...Valentin De Bortoli

Lec15 graph laplacian embeddingUnited States Air Force Academy

Viewers also liked (20)

NIPS 2016. BayesOpt workshop invited talk.Joshua Knowles

An EducationJoshua Knowles

Towards billion bit optimization via parallel estimation of distribution algo...kknsastry

The document describes research into using efficient estimation of distribution algorithms (EDAs) like the compact genetic algorithm (cGA) to solve optimization problems involving billions of variables. Key aspects discussed include the cGA's memory and computational efficiency through techniques like parallelization and vectorization. The researchers were able to solve a noisy OneMax problem involving over 33 million variables to optimality and a problem with 1.1 billion variables with relaxed convergence. The document argues this research is important because many real-world problems involving nanotechnology, biology, and information systems require solving optimization problems at massive scales.

Effects of a Deterministic Hill climber on hBOAMartin Pelikan

Hybridization of global and local search algorithms is a well-established technique for enhancing the efficiency of search algorithms. Hybridizing estimation of distribution algorithms (EDAs) has been repeatedly shown to produce better performance than either the global or local search algorithm alone. The hierarchical Bayesian optimization algorithm (hBOA) is an advanced EDA which has previously been shown to benefit from hybridization with a local searcher. This paper examines the effects of combining hBOA with a deterministic hill climber (DHC). Experiments reveal that allowing DHC to find the local optima makes model building and decision making much easier for hBOA. This reduces the minimum population size required to find the global optimum, which substantially improves overall performance.

Analysis of Evolutionary Algorithms on the One-Dimensional Spin Glass with Po...Martin Pelikan

The document analyzes evolutionary algorithms for solving instances of the one-dimensional spin glass problem with power-law interactions. It generates 610,000 problem instances with varying system size (n=20-150) and power-law decay parameter (σ=0-2). It compares the performance of genetic algorithms, hierarchical Bayesian optimization, and local search methods on these instances, measuring the number of evaluations required to find the optimal solution. The results show how performance depends on the power-law decay and effective interaction range.

iBOA: The Incremental Bayesian Optimization AlgorithmMartin Pelikan

This paper proposes the incremental Bayesian optimization algorithm (iBOA), which modifies standard BOA by removing the population of solutions and using incremental updates of the Bayesian network. iBOA is shown to be able to learn and exploit unrestricted Bayesian networks using incremental techniques for updating both the structure as well as the parameters of the probabilistic model. This represents an important step toward the design of competent incremental estimation of distribution algorithms that can solve difficult nearly decomposable problems scalably and reliably.

Using Previous Models to Bias Structural Learning in the Hierarchical BOAMartin Pelikan

Estimation of distribution algorithms (EDAs) are stochastic optimization techniques that explore the space of potential solutions by building and sampling explicit probabilistic models of promising candidate solutions. While the primary goal of applying EDAs is to discover the global optimum or at least its accurate approximation, besides this, any EDA provides us with a sequence of probabilistic models, which in most cases hold a great deal of information about the problem. Although using problem-specific knowledge has been shown to significantly improve performance of EDAs and other evolutionary algorithms, this readily available source of problem-specific information has been practically ignored by the EDA community. This paper takes the first step towards the use of probabilistic models obtained by EDAs to speed up the solution of similar problems in future. More specifically, we propose two approaches to biasing model building in the hierarchical Bayesian optimization algorithm (hBOA) based on knowledge automatically learned from previous hBOA runs on similar problems. We show that the proposed methods lead to substantial speedups and argue that the methods should work well in other applications that require solving a large number of problems with similar structure.

Fitness inheritance in the Bayesian optimization algorithmMartin Pelikan

This paper describes how fitness inheritance can be used to estimate fitness for a proportion of newly sampled candidate solutions in the Bayesian optimization algorithm (BOA). The goal of estimating fitness for some candidate solutions is to reduce the number of fitness evaluations for problems where fitness evaluation is expensive. Bayesian networks used in BOA to model promising solutions and generate the new ones are extended to allow not only for modeling and sampling candidate solutions, but also for estimating their fitness. The results indicate that fitness inheritance is a promising concept in BOA, because population-sizing requirements for building appropriate models of promising solutions lead to good fitness estimates even if only a small proportion of candidate solutions is evaluated using the actual fitness function. This can lead to a reduction of the number of actual fitness evaluations by a factor of 30 or more.

Spurious Dependencies and EDA ScalabilityMartin Pelikan

Empirical Analysis of ideal recombination on random decomposable problemskknsastry

This paper analyzes the behavior of a selectorecombinative genetic algorithm (GA) with an ideal crossover on a class of random additively decomposable problems (rADPs). Specifically, additively decomposable problems of order k whose subsolution fitnesses are sampled from the standard uniform distribution U[0,1] are analyzed. The scalability of the selectorecombinative GA is investigated for 10,000 rADP instances. The validity of facetwise models in bounding the population size, run duration, and the number of function evaluations required to successfully solve the problems is also verified. Finally, rADP instances that are easiest and most difficult are also investigated.

Intelligent Bias of Network Structures in the Hierarchical BOAMartin Pelikan

This document describes research on intelligently biasing the network structures learned by the hierarchical Bayesian Optimization Algorithm (hBOA) to improve its performance. The researchers developed a method called split probability matrix (SPM) biasing, which uses prior information stored in an SPM to bias hBOA's network building process towards structures with a certain number of splits. They tested SPM biasing on two problems: Trap-5 and 2D Ising spin glasses. The experiments showed that SPM biasing significantly reduced hBOA's execution time, number of evaluations, and bits examined on both problems, with the best speedups achieved around a tuning parameter κ value of 1.

Simplified Runtime Analysis of Estimation of Distribution AlgorithmsPer Kristian Lehre

Efficiency Enhancement of Estimation of Distribution AlgorithmsMartin Pelikan

This document discusses enhancing the efficiency of estimation of distribution algorithms (EDAs). EDAs guide search by building and sampling an explicit probabilistic model of high-quality solutions and can solve hard problems scalably. However, for very large problems with complex evaluations, their performance may not be sufficient. The authors from the University of Missouri propose methods for improving the efficiency of EDAs.

Transfer Learning, Soft Distance-Based Bias, and the Hierarchical BOAMartin Pelikan

An automated technique has recently been proposed to transfer learning in the hierarchical Bayesian optimization algorithm (hBOA) based on distance-based statistics. The technique enables practitioners to improve hBOA efficiency by collecting statistics from probabilistic models obtained in previous hBOA runs and using the obtained statistics to bias future hBOA runs on similar problems. The purpose of this paper is threefold: (1) test the technique on several classes of NP-complete problems, including MAXSAT, spin glasses and minimum vertex cover; (2) demonstrate that the technique is effective even when previous runs were done on problems of different size; (3) provide empirical evidence that combining transfer learning with other efficiency enhancement techniques can often yield nearly multiplicative speedups.

Initial-Population Bias in the Univariate Estimation of Distribution AlgorithmMartin Pelikan

This document studies the effects of biasing the initial population in the Univariate Marginal Distribution Algorithm (UMDA) on the onemax and noisy onemax problems. Theoretical models are developed to predict the impact on population size, number of generations, and number of evaluations for different levels of initial bias. Experiments match the theoretical predictions, showing that a positively biased initial population improves performance while a negatively biased population harms performance. Introducing noise does not change these effects.

Using Problem-Specific Knowledge and Learning from Experience in Estimation o...Martin Pelikan

Analyzing Probabilistic Models in Hierarchical BOA on Traps and Spin GlassesMartin Pelikan

The hierarchical Bayesian optimization algorithm (hBOA) can solve nearly decomposable and hierarchical problems of bounded difficulty in a robust and scalable manner by building and sampling probabilistic models of promising solutions. This paper analyzes probabilistic models in hBOA on two common test problems: concatenated traps and 2D Ising spin glasses with periodic boundary conditions. We argue that although Bayesian networks with local structures can encode complex probability distributions, analyzing these models in hBOA is relatively straightforward and the results of such analyses may provide practitioners with useful information about their problems. The results show that the probabilistic models in hBOA closely correspond to the structure of the underlying optimization problem, the models do not change significantly in subsequent iterations of BOA, and creating adequate probabilistic models by hand is not straightforward even with complete knowledge of the optimization problem.

Order Or Not: Does Parallelization of Model Building in hBOA Affect Its Scala...Martin Pelikan

1. Martin Pelikan and James D. Laury Jr. study the effects of parallelizing the model building process in hBOA on its scalability. 2. They find that parallelizing model building provides nearly linear speedups but can slightly increase the number of function evaluations needed. 3. However, the negative effects of parallelizing model building are negligible compared to the performance gains, so parallelizing model building is beneficial when it is the bottleneck.

Estimation of Distribution Algorithms TutorialMartin Pelikan

Probabilistic model-building algorithms (PMBGAs), also called estimation of distribution algorithms (EDAs) and iterated density estimation algorithms (IDEAs), replace traditional variation of genetic and evolutionary algorithms by (1) building a probabilistic model of promising solutions and (2) sampling the built model to generate new candidate solutions. PMBGAs are also known as estimation of distribution algorithms (EDAs) and iterated density-estimation algorithms (IDEAs). Replacing traditional crossover and mutation operators by building and sampling a probabilistic model of promising solutions enables the use of machine learning techniques for automatic discovery of problem regularities and exploitation of these regularities for effective exploration of the search space. Using machine learning in optimization enables the design of optimization techniques that can automatically adapt to the given problem. There are many successful applications of PMBGAs, for example, Ising spin glasses in 2D and 3D, graph partitioning, MAXSAT, feature subset selection, forest management, groundwater remediation design, telecommunication network design, antenna design, and scheduling. This tutorial provides a gentle introduction to PMBGAs with an overview of major research directions in this area. Strengths and weaknesses of different PMBGAs will be discussed and suggestions will be provided to help practitioners to choose the best PMBGA for their problem. The video of this tutorial presented at GECCO-2008 can be found at http://medal.cs.umsl.edu/blog/?p=293

Slideshare pptMandy Suzanne