Introduction

Framework Description
Application

A Discrepancy-based Framework to Compare Robustness

between Multi-Attribute Evaluations

J. Raimbault1,2
[email protected]

1 UMR CNRS 8504 Géographie-cités

2 UMR-T IFSTTAR 9403 LVMT

CSD&M 2016 - Paris

14th December 2016
 Introduction
Context
Framework Description
Research Question
Application

Multi-objective Complex Systems

From morphogenetic factors to optimal design, fundamental

multi-objective nature of optimization processes

Source : Left [Raimbault et al., 2014] ; Right [Raimbault and Gonzalez, 2015]
 Introduction
Context
Framework Description
Research Question
Application

Multi-objective Evaluations of Socio-technical Systems

Systematic multi-objective nature of problems in design of Complex In-

dustrial Systems [Marler and Arora, 2004] and in the study of Complex
Natural Systems [Newman, 2011]

→ Design and understanding (i.e. evaluation) of Socio-technical systems at

the intersection as hybrid systems [Picon, 2013][Haken and Portugali, 2003]

→ Territorial systems as typical examples : e.g. sustainable urban de-

sign [Souami, 2012], multi-criteria decision-making for transportation in-
frastructures [Bavoux et al., 2005]
 Introduction
Context
Framework Description
Research Question
Application

Robustness of Evaluations

Reliability of an evaluation is crucial; various approaches to robustness

depending on method

→ Naturally included in the construction and estimation of statistical mod-

els [Launer and Wilkinson, 2014] (e.g. p-value, beta power, AIC)

→ In multi-objective optimization, diverse methods: sensitivity of Pareto

front to perturbations [Deb and Gupta, 2006]; continuity of solutions
[Barrico and Antunes, 2006]

→ [Dobbie and Dail, 2013] studies robustness for multi-attribute evalua-

tions as biases depending on weighting techniques
 Introduction
Context
Framework Description
Research Question
Application

Towards a Generic Robustness Framework

Research Objective : Investigate a generic data-driven approach to

Robustness in Multi-attribute evaluations of Complex Socio-Technical Sys-
tems

→ Model-independence and method-independence; framework based only

on data structure (and thus quality) and indicator values
→ Particular case of multi-attribute evaluations, where dimensions are
aggregated, in order to obtain a simple measure of robustness
 Introduction Theoretical Framing
Framework Description Assumptions
Application Formal Description

Intuitive and Theoretical Framing

1 Systems are seen from the perspective of raw data available:

data-driven approach
2 A choice of indicators captures the realization of an “urban
fact” [Mangin and Panerai, 1999], in the sense of stylized process
with different spatial realizations
3 Given many systems and indicators, a common space can be build to
compare them
4 Discrepancy of data [Dick and Pillichshammer, 2010] in that space
captures various aspects linked to robustness: system scale and
range, precision of data, missing data
5 Robustness must also capture an error done on indicator
computation
 Introduction Theoretical Framing
Framework Description Assumptions
Application Formal Description

Assumptions

Objectives as Kernel Integrals

→ Kernel functions exist for each objective, computed as their integrals
→ Reasonable for most systems, as e.g. for territorial systems it is analog to
smoothing of Geographically Weighted Regression [Brunsdon et al., 1998]

Linearly Aggregated Objectives

→ Aggregated objective as linear combination of attributes q(~x) = ∑i wi qi (~x)
→ Choice of weights at the core of decision-making process; not our scope
L = q̂i,c
here, we take simply relative indicator importance wi,c ∑ q̂i,c c
 Introduction Theoretical Framing
Framework Description Assumptions
Application Formal Description

Formal Description (I)

Territorial Systems : Data Si = Xi ∈ Xi with Xi = ∏k Xi,k

System space
!
  X
pi,k
X = ∏ X˜c = ∏ R
def Xi,k ∈DX

Objectives : Hc space of real-valued functions on X˜c , such that :

1 hc ∈ Hc are “enough” regular (tempered distributions e.g.)
R
2 qc = X˜c h is a function describing the “urban fact” (the indicator in
itself)
3 Normalized kernels hc (~x) ∈ [0, 1]
 Introduction Theoretical Framing
Framework Description Assumptions
Application Formal Description

Formal Description (II)

Integral approximation theorem gives upper bound on error, linked to
data discrepancy [Niederreiter, 1972][Varet, 2010]

1
Z
hc −
ni,c ∑ hc (X~ i,c,l ) ≤ K · |||hc ||| · Di,c
l

which propagates to the linear aggregation

1
Z
∑ wi,c hc − ni,c ∑ wi,c hc (X~ i,c,l ) ≤ K ∑ |wi,c | |||hc ||| · Di,c
l c

→ A relative Robustness Ratio can thus be defined between two

evaluations :

∑c wi,c · Di,c
Ri,i 0 = (1)
∑c wi 0 ,c · Di 0 ,c
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Implementation on Synthetic Data

Using OpenStreetMap roads and buildings data, construction of synthetic

indicators on Paris districts (car daily use, car flows in streets, relative
length of pedestrian streets), computed in Monte-carlo simulations
→ Minimal ratio (relative to 1st Arr.) in 15th Arr. (0.92 ± 0.03),
intuitively expected

Example of raw data

 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Metropolitan Segregation

Application to metropolitan Segregation on Ile-de-France, Insee income

data (2011)

Indicators :
Spatial autocorrelation Moran index
1
Dissimilarity index d = ∑ij wij ∑ij wij X̃i − X̃j
Complementary of distribution
 entropy
ε = 1 + log(N) ∑i ∑ Xk · log ∑ XXi k
1 Xi
k k
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Metropolitan Segregation: Indicators

Example of Segregation maps (linking median income with local Moran)

Local Moran
Median 1.71
Income 0.38
0.24
94420 0.16
48923.5 0.1
42612 0.06
39064 0.03
35968 0.01
33265.5 −0.01
30831 −0.04
28226.5 −0.34
25559 no data
22617
15214
no data
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Metropolitan Segregation: Results

Framework Application : robustness comparison between administrative

areas and sensitivity to missing data
0.03
1.00

0.75

0.02

departement

Std Robustness
Robustness

75
92
93
0.50 94

0.01

0.25

0.00

0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.1 0.2 0.3 0.4 0.5
Missing Data Missing Data

(Left) Robustness ratio compared to full region for each area, as a function of
missing data proportion; (Right) Standard deviations
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Discussion

Applicability
→ Application to decision-making procedures : adding robustness as a
dimension
→ Assumptions validity ranges : some indicators may difficultly be viewed
as spatial integrals (as some accessibility measures [Kwan, 1998])
→ Availability of raw data

Further Developments
→ Application to existing open frameworks (e.g. [Tivadar et al., 2014])
→ More general formulation, first to non-linear aggregation (e.g. for Lip-
schitzian functions [Dragomir, 1999])
→ Multi-dimensional measure of robustness using discrepancy along dif-
ferent dimensions
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Conclusion

→ An original data-driven approach to robustness in multi-attribute eval-

uations of complex socio-technical systems; independent of model and
method
→ Future insights from application to diverse system types and fields should
refine the framework
→ Importance of interdisciplinarity (linking here statistics with computa-
tional modeling), crucial to study Complex Systems

- Special thanks to J. Keutchayan (Ecole Polytechnique de Montréal) for suggesting the

original idea of using discrepancy
- All code and data available at https://github.com/JusteRaimbault/RobustnessDiscr
- Paper preprint available [Raimbault, 2016] at http://arxiv.org/abs/1608.00840
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Reserve Slides

Reserve Slides
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Kernel Examples

Typical concrete example of kernels can be :

A mean of rows of Xi,c is computed with h(x) = x · fi,c (x) where fi,c
is the density of the distribution of the assumed underlying variable.
A rate of elements respecting a given condition C ,
h(x) = fi,c (x)χC (x)
For already aggregated variables Y, a Dirac distribution allows to
express them also as a kernel integral.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Implementation

Preprocessing of geographical data is made through QGIS

[QGis, 2011]
Core implementation of the framework is done in R [Team, 2000] for
the flexibility of data management and statistical computations
package DiceDesign[Franco et al., 2009] for computation of
discrepancies
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Synthetic Indicators (I)

Complementary of the average daily distance to work with car per

individual, approximated by, with ncars (b) number of cars in the
building (randomly generated by associated of cars to a number of
building proportional to motorization rate αm 0.4 in Paris), dw
distance to work of individuals (generated from the building to a
uniformly generated random point in spatial extent of the dataset),
and dmax the diameter of Paris area,
d¯w = 1 − |b∈A(a)|
1
· ∑b∈A(a) ncars (b) · ddmax
w
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Synthetic Indicators (II)

Complementary of average car flows within the streets in the district,

approximated by, with ϕ(s) relative flow in street segment s,
generated through the minimum of 1 and a log-normal distribution
adjusted to have 95% of mass smaller than 1 what mimics the
hierarchical distribution of street use (corresponding to betweenness
centrality), and l(s) segment length,
1
ϕ̄ = 1 − |s∈A(a)| · ∑s∈A(a) ϕ(s) · maxl(s)
(l(s))
Relative length of pedestrian streets p̄, computed through a
randomly uniformly generated dummy variable adjusted to have a
fixed global proportion of segments that are pedestrian.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

Synthetic Numerical Results

 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References I

Barrico, C. and Antunes, C. (2006).

Robustness analysis in multi-objective optimization using a degree of
robustness concept.
In Evolutionary Computation, 2006. CEC 2006. IEEE Congress on,
pages 1887–1892.
Bavoux, J.-J., Beaucire, F., Chapelon, L., and Zembri, P. (2005).
Géographie des transports.
Paris.
Brunsdon, C., Fotheringham, S., and Charlton, M. (1998).
Geographically weighted regression.
Journal of the Royal Statistical Society: Series D (The Statistician),
47(3):431–443.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References II

Deb, K. and Gupta, H. (2006).

Introducing robustness in multi-objective optimization.
Evolutionary Computation, 14(4):463–494.
Dick, J. and Pillichshammer, F. (2010).
Digital nets and sequences: Discrepancy Theory and Quasi–Monte
Carlo Integration.
Cambridge University Press.
Dobbie, M. J. and Dail, D. (2013).
Robustness and sensitivity of weighting and aggregation in
constructing composite indices.
Ecological Indicators, 29:270–277.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References III

Dragomir, S. (1999).
The ostrowski’s integral inequality for lipschitzian mappings and
applications.
Computers & Mathematics with Applications, 38(11):33–37.
Franco, J., Dupuy, D., Roustant, O., and Jourdan, A. (2009).
Dicedesign-package.
Designs of Computer Experiments, page 2.
Haken, H. and Portugali, J. (2003).
The face of the city is its information.
Journal of Environmental Psychology, 23(4):385–408.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References IV

Kwan, M.-P. (1998).

Space-time and integral measures of individual accessibility: a
comparative analysis using a point-based framework.
Geographical analysis, 30(3):191–216.
Launer, R. L. and Wilkinson, G. N. (2014).
Robustness in statistics.
Academic Press.
Mangin, D. and Panerai, P. (1999).
Projet urbain.
Parenthèses.
Marler, R. T. and Arora, J. S. (2004).
Survey of multi-objective optimization methods for engineering.
Structural and multidisciplinary optimization, 26(6):369–395.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References V

Newman, M. (2011).
Complex systems: A survey.
arXiv preprint arXiv:1112.1440.
Niederreiter, H. (1972).
Discrepancy and convex programming.
Annali di matematica pura ed applicata, 93(1):89–97.
Picon, A. (2013).
Smart cities: théorie et critique d’un idéal auto-réalisateur.
B2.
QGis, D. (2011).
Quantum gis geographic information system.
Open Source Geospatial Foundation Project.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References VI

Raimbault, J. (2016).
A discrepancy-based framework to compare robustness between
multi-attribute evaluations.
Forthcoming in Proceedings of CSDM 2016. arXiv preprint
arXiv:1608.00840.
Raimbault, J., Banos, A., and Doursat, R. (2014).
A hybrid network/grid model of urban morphogenesis and
optimization.
In Proceedings of the 4th International Conference on Complex
Systems and Applications (ICCSA 2014), June 23-26, 2014,
Université de Normandie, Le Havre, France; M. A. Aziz-Alaoui, C.
Bertelle, X. Z. Liu, D. Olivier, eds.: pp. 51-60.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References VII

Raimbault, J. and Gonzalez, J. (May 2015).

Application de la morphogénèse de réseaux biologiques à la
conception optimale d’infrastructures de transport.
In Rencontres du Labex Dynamites.
Souami, T. (2012).
Ecoquartiers: secrets de fabrication.
Scrineo.
Team, R. C. (2000).
R language definition.
Tivadar, M., Schaeffer, Y., Torre, A., and Bray, F. (2014).
Oasis–un outil d’analyse de la ségrégation et des inégalités spatiales.
Cybergeo: European Journal of Geography.
 Introduction Synthetic Data
Framework Description Metropolitan Segregation
Application Discussion

References VIII

Varet, S. (2010).
Développement de méthodes statistiques pour la prédiction d’un
gabarit de signature infrarouge.
PhD thesis, Université Paul Sabatier-Toulouse III.