Towards billion bit optimization via parallel estimation of distribution algorithm

Jul 14, 20073 likes719 views

The document describes research into using efficient estimation of distribution algorithms (EDAs) like the compact genetic algorithm (cGA) to solve optimization problems involving billions of variables. Key aspects discussed include the cGA's memory and computational efficiency through techniques like parallelization and vectorization. The researchers were able to solve a noisy OneMax problem involving over 33 million variables to optimality and a problem with 1.1 billion variables with relaxed convergence. The document argues this research is important because many real-world problems involving nanotechnology, biology, and information systems require solving optimization problems at massive scales.

Towards billion bit optimization via parallel estimation of distribution algorithm

This paper analyzes the behavior of a selectorecombinative genetic algorithm (GA) with an ideal crossover on a class of random additively decomposable problems (rADPs). Specifically, additively decomposable problems of order k whose subsolution fitnesses are sampled from the standard uniform distribution U[0,1] are analyzed. The scalability of the selectorecombinative GA is investigated for 10,000 rADP instances. The validity of facetwise models in bounding the population size, run duration, and the number of function evaluations required to successfully solve the problems is also verified. Finally, rADP instances that are easiest and most difficult are also investigated.

Mixed Integer Programming: Analyzing 12 Years of ProgressIBM Decision Optimization

Ad24210214IJERA Editor

This document summarizes a technique called CADU (collaborative adaptive down-sampling and upconversion) to improve image compression at low bit rates. The technique adaptively decreases high frequency information by directionally prefiltering an image before uniform downsampling. This allows the downsampled image to be conventionally compressed while avoiding aliasing artifacts. At the decoder, the low-resolution image is decompressed and then upconverted to the original resolution using constrained least squares restoration with an autoregressive model. Experimental results show CADU outperforms JPEG2000 in PSNR and visual quality at low to medium bit rates. The technique suggests oversampling wastes resources and could hurt quality given tight bit budgets.

Development of Reliability Analysis and Multidisciplinary Design Optimization...Altair

This document summarizes the capabilities of the RAMDO software for reliability-based design optimization (RBDO). It discusses both sensitivity-based and sampling-based RBDO approaches in RAMDO. It also provides examples of multidisciplinary applications of RAMDO in areas like fatigue analysis, casting process design, vehicle crashworthiness, and more. Several published case studies demonstrate how RAMDO has been used to optimize designs while accounting for input variability and reliability constraints.

Concurrent Root Cut Loops to Exploit Random Performance VariabilityIBM Decision Optimization

1 icom 2001Isabault

This study examined two hypotheses about what drives performance on working memory span tasks: the resource-sharing hypothesis and the temporal decay hypothesis. The researchers developed a new paradigm to test these hypotheses. In one experiment, they found that equating duration while manipulating cognitive cost did not significantly change span performance across tasks. A second experiment showed that a continuous operation span task requiring sustained attention had a lower span than a reading operation span task. This supports the idea that working memory span depends more on the attentional demands of the processing task than on duration alone.

4 icwm 2004Isabault

The document summarizes the time-based resource-sharing model of working memory. The model proposes that (1) processing and memory maintenance both require attention, which is limited, requiring sharing. (2) When attention is switched away from maintenance, activation decays over time. (3) Memory retrievals have the most detrimental effect on concurrent maintenance due to a bottleneck. (4) Sharing attention is time-based, with cognitive load determined by the proportion of time attention is captured by processing versus refreshing memory. Experiments support cognitive load varying with retrieval rate and duration rather than complexity.

3 escop 2005Isabault

1. The document summarizes the Time-Based Resource-Sharing model of working memory which proposes that working memory span tasks depend only on the time during which processing captures attention, not on the complexity of the processing. 2. According to the model, working memory performance depends on the rate of processing (number of retrievals per unit time), the duration of individual processing steps, and whether processing involves retrievals from long-term memory. 3. Several experiments are described that support the model by showing equivalent effects of different tasks when their actual processing durations are equated.

R-NSGAIIpaskorn

The document proposes an approach called R-NSGA-II that uses reference points to guide an evolutionary multi-objective optimization algorithm towards regions of interest identified by a decision maker. It modifies NSGA-II, a popular multi-objective evolutionary algorithm, to focus on finding Pareto optimal solutions close to user-provided reference points. The crowding distance metric is changed to measure distance from solutions to reference points instead of from other solutions. This helps produce a set of non-dominated solutions near the regions specified as interesting by the decision maker.

Lift-and-Project Cuts in CPLEX 12.5.1IBM Decision Optimization

Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...SKIM

Download this presentation on our website https://goo.gl/EVrXiy We described an approach that fixes the upper level prior based on the data (hence Empirical Bayes) and cross-validation of a scale parameter. This fixed prior eliminates MCMC iterations, making it much faster than standard HB. The Empirical Bayes approach also avoids issues of inadequate convergence and improper scaling that may occur with MCMC.

Design of Optimal Linear Phase FIR High Pass Filter using Improved Particle S...IDES Editor

This paper presents a novel approach for designing a linear phase digital high pass FIR filter using Improved Particle Swarm Optimization (IPSO) algorithm. Design of FIR filter is a multi-modal optimization problem. The conservative gradient based optimization techniques are not efficient for digital filter design. Given the specifications for the filters to be realized, IPSO algorithm generates a set of optimal filter coefficients and tries to meet the ideal frequency response characteristics. This paper presents the realization of the optimal FIR high pass filter of filter order 20 as per given problem statements. The simulation results have been compared to those obtained from well accepted classical algorithms like Park and McClellan algorithm (PM), and evolutionary algorithms like genetic algorithm (GA) and particle swarm optimization (PSO). The results rationalize that the proposed optimal filter design approach using IPSO outperforms PM, RGA, PSO in the accuracy of the designed filter, as well as in the convergence speed and solution quality

Phenomenological Decomposition Heuristics for Process Design Synthesis of Oil...Alkis Vazacopoulos

The processing of a raw material is a phenomenon that varies its quantity and quality along a specific network and logics and logistics to transform it into final products. To capture the production framework in a mathematical programming model, a full space formulation integrating discrete design variables and quantity-quality relations gives rise to large scale non-convex mixed-integer nonlinear models, which are often difficult to solve. In order to overcome this problem, we propose a phenomenological decomposition heuristic to solve separately in a first stage the quantity and logic variables in a mixed-integer linear model, and in a second stage the quantity and quality variables in a nonlinear programming formulation. By considering different fuel demand scenarios, the problem becomes a two-stage stochastic programming model, where nonlinear models for each demand scenario are iteratively restricted by the process design results. Two examples demonstrate the tailor-made decomposition scheme to construct the complex oil-refinery process design in a quantitative manner.

Winner Determination in Combinatorial Reverse AuctionsShubhashis Shil

This document summarizes a research paper that proposes a new genetic algorithm (GA)-based method called GACRA to solve the winner determination problem in combinatorial reverse auctions. GACRA uses two repair techniques to address infeasible chromosomes and a modified two-point crossover to improve diversity and prevent premature convergence. Experimental results showed that GACRA outperformed an existing GA-based method in terms of processing time and procurement cost.

Profilling ZooESUG

Profiler Zoo introduces several techniques for improving code profiling, such as visualizing test coverage, identifying redundant computation, and using message counting as a proxy for execution time. It presents blueprints for profiling code structure, behavior, and performance bottlenecks. The tool aims to provide more intuitive representations of code profiling compared to traditional approaches.

5 psychonomic 2005 1Isabault

Working memory span tasks involve maintaining items while performing a concurrent processing task. The document summarizes the time-based resource-sharing model, which proposes that working memory spans depend on the rate of processing and duration of the processing component, rather than the complexity of the task. Experiments varying factors like number of retrievals, retrieval speed, and task type support the model by showing spans correlate with actual processing time but not task nature independently of duration. The model argues working memory spans are limited by the time attention is unavailable to refresh memory traces due to concurrent processing.

iBOA: The Incremental Bayesian Optimization AlgorithmMartin Pelikan

This paper proposes the incremental Bayesian optimization algorithm (iBOA), which modifies standard BOA by removing the population of solutions and using incremental updates of the Bayesian network. iBOA is shown to be able to learn and exploit unrestricted Bayesian networks using incremental techniques for updating both the structure as well as the parameters of the probabilistic model. This represents an important step toward the design of competent incremental estimation of distribution algorithms that can solve difficult nearly decomposable problems scalably and reliably.

Analysis of Evolutionary Algorithms on the One-Dimensional Spin Glass with Po...Martin Pelikan

The document analyzes evolutionary algorithms for solving instances of the one-dimensional spin glass problem with power-law interactions. It generates 610,000 problem instances with varying system size (n=20-150) and power-law decay parameter (σ=0-2). It compares the performance of genetic algorithms, hierarchical Bayesian optimization, and local search methods on these instances, measuring the number of evaluations required to find the optimal solution. The results show how performance depends on the power-law decay and effective interaction range.

Simplified Runtime Analysis of Estimation of Distribution AlgorithmsPer Kristian Lehre

Efficiency Enhancement of Estimation of Distribution AlgorithmsMartin Pelikan

This document discusses enhancing the efficiency of estimation of distribution algorithms (EDAs). EDAs guide search by building and sampling an explicit probabilistic model of high-quality solutions and can solve hard problems scalably. However, for very large problems with complex evaluations, their performance may not be sufficient. The authors from the University of Missouri propose methods for improving the efficiency of EDAs.

Intelligent Bias of Network Structures in the Hierarchical BOAMartin Pelikan

This document describes research on intelligently biasing the network structures learned by the hierarchical Bayesian Optimization Algorithm (hBOA) to improve its performance. The researchers developed a method called split probability matrix (SPM) biasing, which uses prior information stored in an SPM to bias hBOA's network building process towards structures with a certain number of splits. They tested SPM biasing on two problems: Trap-5 and 2D Ising spin glasses. The experiments showed that SPM biasing significantly reduced hBOA's execution time, number of evaluations, and bits examined on both problems, with the best speedups achieved around a tuning parameter κ value of 1.

Using Previous Models to Bias Structural Learning in the Hierarchical BOAMartin Pelikan

Estimation of distribution algorithms (EDAs) are stochastic optimization techniques that explore the space of potential solutions by building and sampling explicit probabilistic models of promising candidate solutions. While the primary goal of applying EDAs is to discover the global optimum or at least its accurate approximation, besides this, any EDA provides us with a sequence of probabilistic models, which in most cases hold a great deal of information about the problem. Although using problem-specific knowledge has been shown to significantly improve performance of EDAs and other evolutionary algorithms, this readily available source of problem-specific information has been practically ignored by the EDA community. This paper takes the first step towards the use of probabilistic models obtained by EDAs to speed up the solution of similar problems in future. More specifically, we propose two approaches to biasing model building in the hierarchical Bayesian optimization algorithm (hBOA) based on knowledge automatically learned from previous hBOA runs on similar problems. We show that the proposed methods lead to substantial speedups and argue that the methods should work well in other applications that require solving a large number of problems with similar structure.

Fitness inheritance in the Bayesian optimization algorithmMartin Pelikan

This paper describes how fitness inheritance can be used to estimate fitness for a proportion of newly sampled candidate solutions in the Bayesian optimization algorithm (BOA). The goal of estimating fitness for some candidate solutions is to reduce the number of fitness evaluations for problems where fitness evaluation is expensive. Bayesian networks used in BOA to model promising solutions and generate the new ones are extended to allow not only for modeling and sampling candidate solutions, but also for estimating their fitness. The results indicate that fitness inheritance is a promising concept in BOA, because population-sizing requirements for building appropriate models of promising solutions lead to good fitness estimates even if only a small proportion of candidate solutions is evaluated using the actual fitness function. This can lead to a reduction of the number of actual fitness evaluations by a factor of 30 or more.

Effects of a Deterministic Hill climber on hBOAMartin Pelikan

Hybridization of global and local search algorithms is a well-established technique for enhancing the efficiency of search algorithms. Hybridizing estimation of distribution algorithms (EDAs) has been repeatedly shown to produce better performance than either the global or local search algorithm alone. The hierarchical Bayesian optimization algorithm (hBOA) is an advanced EDA which has previously been shown to benefit from hybridization with a local searcher. This paper examines the effects of combining hBOA with a deterministic hill climber (DHC). Experiments reveal that allowing DHC to find the local optima makes model building and decision making much easier for hBOA. This reduces the minimum population size required to find the global optimum, which substantially improves overall performance.

Spurious Dependencies and EDA ScalabilityMartin Pelikan

Transfer Learning, Soft Distance-Based Bias, and the Hierarchical BOAMartin Pelikan

An automated technique has recently been proposed to transfer learning in the hierarchical Bayesian optimization algorithm (hBOA) based on distance-based statistics. The technique enables practitioners to improve hBOA efficiency by collecting statistics from probabilistic models obtained in previous hBOA runs and using the obtained statistics to bias future hBOA runs on similar problems. The purpose of this paper is threefold: (1) test the technique on several classes of NP-complete problems, including MAXSAT, spin glasses and minimum vertex cover; (2) demonstrate that the technique is effective even when previous runs were done on problems of different size; (3) provide empirical evidence that combining transfer learning with other efficiency enhancement techniques can often yield nearly multiplicative speedups.

Initial-Population Bias in the Univariate Estimation of Distribution AlgorithmMartin Pelikan

This document studies the effects of biasing the initial population in the Univariate Marginal Distribution Algorithm (UMDA) on the onemax and noisy onemax problems. Theoretical models are developed to predict the impact on population size, number of generations, and number of evaluations for different levels of initial bias. Experiments match the theoretical predictions, showing that a positively biased initial population improves performance while a negatively biased population harms performance. Introducing noise does not change these effects.

Using Problem-Specific Knowledge and Learning from Experience in Estimation o...Martin Pelikan

The Bayesian Optimization Algorithm with Substructural Local SearchMartin Pelikan

This work studies the utility of using substructural neighborhoods for local search in the Bayesian optimization algorithm (BOA). The probabilistic model of BOA, which automatically identifies important problem substructures, is used to define the structure of the neighborhoods used in local search. Additionally, a surrogate fitness model is considered to evaluate the improvement of the local search steps. The results show that performing substructural local search in BOA significatively reduces the number of generations necessary to converge to optimal solutions and thus provides substantial speedups.

Analyzing Probabilistic Models in Hierarchical BOA on Traps and Spin GlassesMartin Pelikan

The hierarchical Bayesian optimization algorithm (hBOA) can solve nearly decomposable and hierarchical problems of bounded difficulty in a robust and scalable manner by building and sampling probabilistic models of promising solutions. This paper analyzes probabilistic models in hBOA on two common test problems: concatenated traps and 2D Ising spin glasses with periodic boundary conditions. We argue that although Bayesian networks with local structures can encode complex probability distributions, analyzing these models in hBOA is relatively straightforward and the results of such analyses may provide practitioners with useful information about their problems. The results show that the probabilistic models in hBOA closely correspond to the structure of the underlying optimization problem, the models do not change significantly in subsequent iterations of BOA, and creating adequate probabilistic models by hand is not straightforward even with complete knowledge of the optimization problem.

More Related Content

What's hot (8)

R-NSGAIIpaskorn

Lift-and-Project Cuts in CPLEX 12.5.1IBM Decision Optimization

Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...SKIM

Design of Optimal Linear Phase FIR High Pass Filter using Improved Particle S...IDES Editor

Phenomenological Decomposition Heuristics for Process Design Synthesis of Oil...Alkis Vazacopoulos

Winner Determination in Combinatorial Reverse AuctionsShubhashis Shil

Profilling ZooESUG

5 psychonomic 2005 1Isabault

R-NSGAIIpaskorn

Lift-and-Project Cuts in CPLEX 12.5.1IBM Decision Optimization

Iterative Multilevel Empirical Bayes (IMEB): An efficient flexible and robust...SKIM

Design of Optimal Linear Phase FIR High Pass Filter using Improved Particle S...IDES Editor

Phenomenological Decomposition Heuristics for Process Design Synthesis of Oil...Alkis Vazacopoulos

Winner Determination in Combinatorial Reverse AuctionsShubhashis Shil

Profilling ZooESUG

5 psychonomic 2005 1Isabault

Viewers also liked (16)