Graph Signal Processing: Fundamentals and

Applications to Diffusion Processes

Antonio G. Marques† , Santiago Segarra‡ , Alejandro Ribeiro?

†
King Juan Carlos University
‡
Massachusetts Institute of Technology
?
University of Pennsylvania
[email protected]
Thanks: Gonzalo Mateos, Geert Leus, and Weiyu Huang
Spanish MINECO grant No TEC2013-41604-R

EUSIPCO16
Budapest, Hungary - August 29, 2016
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 1 / 118
Network Science analytics

Online social media Internet Clean energy and grid analy,cs

I Desiderata: Process, analyze and learn from network data [Kolaczyk’09]

I Network as graph G = (V, E, W ): encode pairwise relationships

I Interest here not in G itself, but in data associated with nodes in V

⇒ Object of study is a graph signal x

I Q: Graph signals common and interesting as networks are?

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 2 / 118
Network of economic sectors of the United States

I Bureau of Economic Analysis of the U.S. Department of Commerce

I E = Output of sector i is an input to sector j (62 sectors in V)

Oil and Gas Services Finance

PC AS FR RA

OG IC

CO MP SC MP

I Oil extraction (OG), Petroleum and coal products (PC), Construction (CO)
I Administrative services (AS), Professional services (MP)
I Credit intermediation (FR), Securities (SC), Real state (RA), Insurance (IC)

I Only interactions stronger than a threshold are shown

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 3 / 118

Network of economic sectors of the United States

I Bureau of Economic Analysis of the U.S. Department of Commerce

I E = Output of sector i is an input to sector j (62 sectors in V)

I A few sectors have widespread

strong influence (services,
finance, energy)
I Some sectors have strong
indirect influences (oil)
I The heavy last row is final
consumption

I This is an interesting network ⇒ Signals on this graph are as well

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 4 / 118

Disaggregated GDP of the United States

I Signal x = output per sector = disaggregated GDP

⇒ Network structure used to, e.g., reduce GDP estimation noise

I Signal is as interesting as the network itself. Arguably more

I Same is true on brain connectivity and fMRI brain signals, ...
I Gene regulatory networks and gene expression levels, ...
I Online social networks and information cascades, ...
I Alignment of customer preferences and product ratings, ...

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 5 / 118

Graph signal processing

I Graph SP: broaden classical SP to graph signals [Shuman et al.’13]

⇒ Our view: GSP well suited to study network (diffusion) processes

x2 x4
2 4
x1
1

3 5
x3 x5

I As.: Signal properties related to topology of G (locality, smoothness)

⇒ Algorithms that fruitfully leverage this relational structure

I Q: Why do we expect the graph structure to be useful in processing x?

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 6 / 118

Importance of signal structure in time

I Signal and Information Processing is about exploiting signal structure

x1
1
I Discrete time described by cyclic graph x6 x2
6 2
⇒ Time n follows time n − 1
⇒ Signal value xn similar to xn−1 x5 x3
5 3
I Formalized with the notion of frequency
4
x4

e j2πkn/N
 
H
I Cyclic structure ⇒ Fourier transform ⇒ x̃ = F x Fkn = √
N
I Fourier transform ⇒ Projection on eigenvector space of cycle

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 7 / 118

Covariances and principal components

 
I Random signal with mean E [x] = 0 and covariance Cx = E xxH
⇒ Eigenvector decomposition Cx = VΛVH
x1
σ16 1 σ12
I Covariance matrix Cx is a graph x6 x2
6 2
⇒ Not a very good graph, but still σ15 σ13

I Precision matrix C−1

x a common graph too x5 x3
5 3
⇒ Conditional dependencies of Gaussian x
σ45 4 σ43
x4

I Covariance matrix structure ⇒ Principal components (PCA) ⇒ x̃ = VH x

I PCA transform ⇒ Projection on eigenvector space of (inverse) covariance

I Q: Can we extend these principles to general graphs and signals?

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 8 / 118

Graphs 101

I Formally, a graph G (or a network) is a triplet (V, E, W )

I V = {1, 2, . . . , N} is a finite set of N nodes or vertices

I E ⊆ V × V is a set of edges defined as ordered pairs (n, m)
I Write N (n) = {m ∈ V : (m, n) ∈ E} as the in-neighbors of n

I W : E → R is a map from the set of edges to scalar values wnm

I Represents the level of relationship from n to m
I Often weights are strictly positive, W : E → R++

I Unweighted graphs ⇒ wnm ∈ {0, 1}, for all (n, m) ∈ E

I Undirected graphs ⇒ (n, m) ∈ E if and only if (m, n) ∈ E and
wnm = wmn , for all (n, m) ∈ E

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 9 / 118

Graphs – examples

I Unweighted and directed graphs (e.g., time)

0 1 2 3 ··· 23
I V = {0, 1, . . . , 23}
I E = {(0, 1), (1, 2), . . . , (22, 23), (23, 0)}
I W : (n, m) 7→ 1, for all (n, m) ∈ E

I Unweighted and undirected graphs (e.g., image) 1 2 3

I V = {1, 2, 3, . . . , 9}
4 5 6
I E = {(1, 2), (2, 3), . . . , (8, 9), (1, 4), . . . , (6, 9)}
I W : (n, m) 7→ 1, for all (n, m) ∈ E
7 8 9

Σ11 Σ22
Σ12
p1 p2
I Weighted and undirected graphs (e.g., covariance)
Σ14 I V = {1, 2, 3, 4}
Σ13 Σ23 Σ24
I E = {(1, 1), (1, 2), . . . , (4, 4)} = V × V
p3 p4
I W : (n, m) 7→ σnm = σmn , for all (n, m)
Σ34
Σ33 Σ44

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 10 / 118

Adjacency matrix

I Algebraic graph theory: matrices associated with a graph G

⇒ Adjacency A and Laplacian L matrices
⇒ Spectral graph theory: properties of G using spectrum of A or L

I Given G = (V, E, W ), the adjacency matrix A ∈ RN×N is

(
wnm , if (n, m) ∈ E
Anm =
0, otherwise

I Matrix representation incorporating all information about G

⇒ For unweighted graphs, positive entries represent connected pairs
⇒ For weighted graphs, also denote proximities between pairs

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 11 / 118

Degree and k-hop neighbors

I If G is unweighted and undirected, the degree of node i is |N (i)|

⇒ In directed graphs, have out-degree and an in-degree

I Using the adjacency matrix in the undirected case

P P
⇒ For node i: deg(i) = j∈N (i) Aij = j Aij
⇒ For all N nodes: d = A1 → Degree matrix: D := diag(d)

I Q: Can this be extended to k-hop neighbors? → Powers of A

⇒ [Ak ]ij non-zero only if there exists a path of length k from i to j
⇒ Support of Ak : pairs that can be reached in k hops

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 12 / 118

Laplacian of a graph

I Given undirected G with A and D, the Laplacian matrix L ∈ RN×N is

L=D−A

⇒ Equivalently, L can be defined element-wise as


 deg(i), if i = j
Lij = −w ij , if (i, j) ∈ E
0, otherwise


I Normalized Laplacian: L = D−1/2 LD−1/2 (we will focus on L)

2
 
1 3 3 −1 0 −2
−1 6 −3 −2
1 2 3 L= 
0 −3 4 −1
2 1 −2 −2 −1 5
4

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 13 / 118

Spectral properties of the Laplacian

I Denote by λi and vi the eigenvalues and eigenvectors of L

I L is positive semi-definite
⇒ xT Lx = 12 (i,j)∈E wij (xi − xj )2 ≥ 0, for all x
P

⇒ All eigenvalues are nonnegative, i.e. λi ≥ 0 for all i

I A constant vector 1 is an eigenvector of L with eigenvalue 0

X
[L1]i = wij (1 − 1) = 0
j∈N (i)
√
⇒ Thus, λ1 = 0 and v1 = (1/ N) 1

I In connected graphs, it holds that λi > 0 for i = 2, . . . , N

⇒ Multiplicity{λ = 0} = number of connected components

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 14 / 118

Part I: Fundamentals

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 15 / 118

Graph signals

I Consider graph G = (V, E, W ). Graph signals are mappings x : V → R

⇒ Defined on the vertices of the graph (data tied to nodes)
Ex: Opinion profile, buffer congestion levels, neural activity, epidemic

I May be represented as a vector x ∈ RN

⇒ xn denotes the signal value at the n-th vertex in V
⇒ Implicit ordering of vertices (same as in A or L)

5    
x0 0.6
x1  0.7
0    
x = x2  = 0.3
6 9    
1 4
 ..   .. 
2 3 .  . 
7 8 x9 0.7

I Data associated with links of G ⇒ Use line graph of G

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 16 / 118
Graph signals – Genetic profiles

I Graphs representing gene-gene interactions

⇒ Each node denotes a single gene (loosely speaking)
⇒ Connected if their coded proteins participate in same metabolism

I Genetic profiles for each patient can be considered as a graph signal

⇒ Signal on each node is 1 if mutated and 0 otherwise
   
0 P
P 1
1 T I 0
T I
   
x1 = 0 C P I
x2 = 0
  C P I
 
 ..   .. 
. K T
K T .
J
J
0 0
Sample patient 2 with subtype 1
Sample patient 1 with subtype 1

I To understand a graph signal, the structure of G must be considered

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 17 / 118

Graph-shift operator

I To understand and analyze x, useful to account for G ’s structure

I Associated with G is the graph-shift operator S ∈ RN×N

⇒ Sij = 0 for i 6= j and (i, j) 6∈ E (captures local structure in G )

I S can take nonzero values in the edges of G or in its diagonal

I Ex: Adjacency A, degree D, and Laplacian L = D − A matrices

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 18 / 118

Relevance of the graph-shift operator

I Q: Why is S called shift? A: Resemblance to time shifts

I S will be building block for GSP algorithms (More soon)

⇒ Same is true in the time domain (filters and delay)

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 19 / 118

Local structure of graph-shift operator

S represents a linear transformation that can be computed locally at

the nodes of the graph. More rigorously, if y is defined as y = Sx,
then node i can compute yi if it has access to xj at j ∈ N (i).

I Straightforward because [S]ij 6= 0 only if i = j or (j, i) ∈ E

I What if y = S2 x?
⇒ Like powers of
A: neighborhoods
⇒ yi found using
values within 2-hops
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 20 / 118
Graph Fourier Transform

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 21 / 118

Discrete Fourier Transform (DFT)

2π
I Let x be a temporal signal, its DFT is x̃ = FH x, with Fkn = √1 e +j N kn
N
⇒ Equivalent description, provides insights
⇒ Oftentimes, more parsimonious (bandlimited)
⇒ Facilitates the design of SP algorithms: e.g., filters

I Many other transformations (orthogonal dictionaries) exist

I Q: What transformation is suitable for graph signals?

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 22 / 118

Graph Fourier Transform (GFT)

I Useful transformation? ⇒ S involved in generation/description of x

⇒ Let S = VΛV−1 be the shift associated with G

I The Graph Fourier Transform (GFT) of x is defined as

x̃ = V−1 x

I While the inverse GFT (iGFT) of x̃ is defined as

x = Vx̃

⇒ Eigenvectors V = [v1 , ..., vN ] are the frequency basis (atoms)

I Additional structure
⇒ If S is normal, then V−1 = VH and x̃k = vH
k x =< vk , x >
⇒ Parseval holds, kxk2 = kx̃k2

I GFT ⇒ Projection on eigenvector space of shift operator S

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 23 / 118
Is this a reasonable transform?

I Particularized to cyclic graphs ⇒ GFT ≡ Fourier transform

I Particularized to covariance matrices ⇒ GFT ≡ PCA transform

I But really, this is an empirical question. GFT of disaggregated GDP

I GFT transform characterized by a few coefficients

PK
⇒ Notion of bandlimitedness: x = k=1 x̃k vk
⇒ Sampling, compression, filtering, pattern recognition

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 24 / 118

Eigenvalues as frequencies
P
I Columns of V are the frequency atoms: x = k x̃k vk
I Q: What about the eigenvalues λk = Λkk
2π
⇒ When S = Adc , we get λk = e −j N k
⇒ λk can be viewed as frequencies!!
I In time, well-defined relation between frequency and variation
⇒ Higher k ⇒ higher oscillations
P 2
⇒ Bounds on total-variation: TV (x) = n (xn − xn−1 )

I Q: Does this carry over for graph signals?

⇒ No in general, but if S = L there are interpretations for λk
⇒ {λk }Nk=1 will be very important when analyzing graph filters
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 25 / 118
Interpretation of the Laplacian

I Consider a graph G , let x be a signal on G , and set S = L

P
⇒ y = Sx is now y = Lx ⇒ yi = j∈N (i) wij (xi − xj )
⇒ j-th term is large if xj is very different from neighboring xi
⇒ yi measures difference of xi relative to its neighborhood

I We can also define the quadratic form xT Sx

1 X
xT Lx = wij (xi − xj )2
2
(i,j)∈E
T
⇒ x Lx quantifies the (aggregated) local variation of signal x
⇒ Natural measure of signal smoothness w.r.t. G

I Q: Interpretation of frequencies {λk }N

k=1 when S = L?
⇒ If x = vk , we get xT Lx = λk ⇒ local variation of vk
⇒ Frequencies account for local variation, they can be ordered
⇒ Eigenvector associated with eigenvalue 0 is constant
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 26 / 118
Frequencies of the Laplacian

I Laplacian eigenvalue λk accounts for the local variation of vk

⇒ Let us plot some of the eigenvectors of L (also graph signals)
I Ex: gene network, N = 10, k = 1, k = 2, k = 9

P P P
T I T I T I

C P I C P I C P I

K T K T K T
J J J

I Ex: smooth natural images, N = 216 , k = 2, ..., 6

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 27 / 118

Application: Cancer subtype classification

I Patients diagnosed with same disease exhibit different behaviors

I Each patient has a genetic profile describing gene mutations

I Would be beneficial to infer phenotypes from genotypes

⇒ Targeted treatments, more suitable suggestions, etc.

I Traditional approaches consider different genes to be independent

⇒ Not ideal, as different genes may affect same metabolism

I Alternatively, consider genetic network

⇒ Genetic profiles become graph signals on genetic network
⇒ We will see how this consideration improves subtype classification

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 28 / 118

Genetic network

I Undirected and unweighted gene-to-gene interaction graph

I 2458 nodes are genes in human DNA related to breast cancer
I An edge between two genes represents interaction
⇒ Coded proteins participate in the same metabolic process

I Adjacency matrix of the gene-interaction network

0

500

1000
Genes

1500

2000

0 500 1000 1500 2000

Genes

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 29 / 118

Genetic profiles

I Genetic profile of 240 women with breast cancer

⇒ 44 with serous subtype and 196 with endometrioid subtype
⇒ Patient i has an associated profile xi ∈ {0, 1}2458
I Mutations are very varied across patients
⇒ Some patients present a lot of mutations
⇒ Some genes are consistently mutated across patients

I Q: Can we use genetic profiles to classify patients across subtypes?

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 30 / 118
Improving k-nearest neighbor classification

I Distance between genetic profiles ⇒ d(i, j) = kxi − xj k2

⇒ N-fold cross-validation error from k-NN classification

k = 3 ⇒ 13.3%, k = 5 ⇒ 12.9%, k = 7 ⇒ 14.6%

I Q: Can we do any better using graph signal processing?
I Each genetic profile xi is a graph signal on the genetic network
⇒ Look at the frequency components x̃i using the GFT
⇒ Use as shift operator S the Laplacian of the genetic network
Example of signal xi Frequency representation x̃i
1 1

0.9 0.9

0.8 0.8

0.7 0.7

0.6 0.6
Amplitude
Signal

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0 0
0 500 1000 1500 2000 2500 0 500 1000 1500 2000 2500
Gene Frequencies

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 31 / 118

Distinguishing Power
I Define the distinguishing power of frequency vk as
P P
i:yi =1 x̃i (k) i:y =2 x̃i (k)
X
DP(vk ) = P −P i / |x̃i (k)| ,
i 1 {yi = 1} i 1 {yi = 2} i
I Normalized difference between the mean GFT coefficient for vk
⇒ Among patients with serous and endometrioid subtypes
I Distinguishing power is not equal across frequencies

0.01

0.009

0.008
Distinguishing Power

0.007

0.006

0.005

0.004

0.003

0.002

0.001

0
0 500 1000 1500 2000 2500
Frequency

I The distinguishing power defined is one of many proper heuristics

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 32 / 118
Increasing accuracy by selecting the best frequencies
I Keep information in frequencies with higher distinguishing power
⇒ Filter, i.e., multiply x̃i by diag(h̃p ) where
(
p 1, if DP(vk ) ≥ p-th percentile of DP
[h̃ ]k =
0, otherwise
I Then perform inverse GFT to get the graph signal x̂i

15
error percentage

10

original
5
one frequency
p = 60
p = 75
p = 90
0
k=3 k=5 k=7

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 33 / 118

Graph filters and network processes

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 34 / 118

Linear (shift-invariant) graph filter

I A graph filter H : RN → RN is a map between graph signals

Focus on linear filters
⇒ map represented by an
N × N matrix

DEF1: Polynomial in S of degree L, with coeff. h = [h0 , . . . , hL ]T

PL
H := h0 S0 + h1 S1 + . . . + hL SL = l=0 hl Sl [Sandryhaila13]

DEF2: Orthogonal operator in the frequency domain

H := Vdiag h̃ V−1 ,


h̃k = g (λk )

I With [Ψ]k,l := λl−1

k , we have h̃ = Ψh ⇒ Defs can be rendered equivalent
⇒ More on this later, now focus on DEF1
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 35 / 118
Graph filters as linear network operators

PL
I DEF1 says H = l=0 hl Sl
I Suppose H acts on a graph signal x to generate y = Hx
⇒ If we define x(l) := Sl x = Sx(l−1)
y is a linear
L
X combination of
y= hl x(l)
successive shifted
l=0
versions of x
I After introducing S, we stressed that y = Sx can be computed locally
⇒ x(l) can be found locally if x(l−1) is known
⇒ The output of the filter can be found in L local steps

I A graph filter represents a linear transformation that

⇒ Accounts for local structure of the graph
⇒ Can be implemented distributedly in L steps
⇒ Only requires info in L-neighborhood [Shuman13, Sandyhaila14]

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 36 / 118

An example of a graph filter
PL PL
I x = [−1, 2, 0, 0, 0, 0]T , h = [1, 1, 0.5]T , y = ( l=0 hl S)x = l=0 hl x(l)

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 37 / 118

Frequency response of a graph filter

I Recalling that S = VΛV−1 , we may write

L L L
!
X X X
−1
H= l
hl S = hl VΛ Vl
=V hl Λ l
V−1
l=0 l=0 l=0

I The application Hx of filter H to x can be split into three parts

⇒ V−1 takes signal x to the graph frequency domain x̃
PL
⇒ H̃ := l=0 hl Λl modifies the frequency coefficients to obtain ỹ
⇒ V brings the signal ỹ back to the graph domain y

I Since H̃ is diagonal, define H̃ =: diag (h̃)

⇒ h̃ is the frequency response of the filter H
⇒ Output at frequency k depends only on input at frequency k

ỹk = h̃k x̃k

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 38 / 118

Frequency response and filter coefficients

I Relation between h̃ and h in a more friendly manner?

PL PL
⇒ Since h̃ = diag( l=0 hl Λl ), we have that h̃k = l=0 hl λlk
⇒ Define the Vandermonde matrix Ψ as
λL1
 
1 λ1 ...
 . .. .. 
Ψ :=  .. . . 
1 λN ... λLN

Frequency response of a graph filter

If h are the coefficients of a graph filter, its frequency response is

h̃ = Ψh

I Given a desired h̃ , we can find the coefficients h as

h = Ψ−1 h̃
⇒ Since Ψ is Vandermonde, invertible as long as λk 6= λk 0 for k 6= k 0
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 39 / 118
More on the frequency response

I Since h = Ψ−1 h̃ ⇒ If all {λk }N

k=1 distinct, then
⇒ Any h̃ can be implemented with at most L+1 = N coefficients

I Since h = Ψh̃ ⇒ If λk = λk 0 , then

⇒ The corresponding frequency response will be the same h̃k = h̃k 0
2π
I For the particular case when S = Adc , we have that λk = e −j N (k−1)

1 1 ... 1
 
2π(1)(1) 2π(1)(N−1)
−j −j
 1 e N ... e N

 = FH
 
Ψ= .. .. ..
. . .
 
 
2π(N−1)(1) 2π(N−1)(N−1)
−j −j
1 e N ... e N

⇒ The frequency response is the DFT of the impulse response

h̃ = FH h

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 40 / 118

Frequency response for graph signals and filters

I Suppose that we have a signal x and filter coefficients h

I For time signals, it holds that the output y is

ỹ = diag(FH h)FH x

I For graph signals, the output y in the frequency domain is

ỹ = diag(Ψh)V−1 x

I The GFT for filters is different from the GFT for signals
⇒ Symmetry is lost, but both depend on spectrum of S
⇒ Many of the properties are not true for graphs
⇒ Several options to generalize operations

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 41 / 118

System identification and impulse response

I Suppose that our goal is to find h given x and y

⇒ Using the previous expressions

h = Ψ−1 diag−1 (V−1 x)V−1 y

I In time, if we set x = [1, 0, ..., 0]T = e1 (i.e., x̃ = 1), we have

⇒ h = Fdiag−1 (1)FH y = y → h is the impulse response

I In the graph domain

I If we set x = ei , then h = Ψ−1 diag−1 (ẽi )V−1 y, where
⇒ ẽi := V−1 ei ≡ how strongly node i expresses each of the freqs.
⇒ Problem if ẽi has zero entries
I Alternatively we can get x̃ = 1 by setting x = V1 and then
⇒ h = Ψ−1 diag−1 (x̃)V−1 y = Ψ−1 V−1 y

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 42 / 118

Implementing graph filters: frequency or space

I Frequency or space?
PL
y = Vdiag(h̃)V−1 x vs. y= l
l=0 hl S x

I In space: leverage the fact that Sx can be computed locally

⇒ Signal x is percolated L times to find {x(l) }Ll=0
PL
⇒ Every node finds its own yi by computing l=0 hl [x(l) ]i
I Frequency implementation useful for processing if, e.g.,
⇒ Filter bandlimited and eigenvectors easy to find
⇒ Low complexity [Anis16, Tremblay16]
I Space definition useful for modeling
⇒ Diffusion, percolation, opinion formation, ... (more on this soon)
I More on filter design
⇒ Chebyshev polyn. [Shuman12]; AR-MA [Isufi-Leus15]; Node-var.
[Segarra15]; Time-var. [Isufi-Leus16]; Median filters [Segarra16]
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 43 / 118
Linear network processes via graph filters

I Consider a linear dynamics of the form

xt − xt−1 = αJxt−1 ⇒ xt = (I − αJ)xt−1

I If x is network process ⇒ [xt ]i depends only on [xt−1 ]j , j ∈ N (i)

[S]ij = [J]ij ⇒ xt = (I − αS)xt−1 ⇒ xt = (I − αS)t x0

⇒ xt = Hx0 , with H a polynomial of S ⇒ linear graph filter

I If the system has memory ⇒ output weighted sum of previous
exchanges (opinion dynamics) ⇒ still a polynomial of S
PT t PT
y= t=0 β xt ⇒ y = t=0 (βI − βαS)t x0

I Everything holds true if αt or βt are time varying

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 44 / 118
Diffusion dynamics and AR (IIR) filters

I Before finite-time dynamics (FIR filters)

I Consider now a diffusion dynamics xt = αSxt−1 + w

Pt 0
xt = αt St x0 + t 0 =0 α
t
St w

⇒ When t → ∞: x∞ = (I − αS)−1 w ⇒ AR graph filter

I Higher orders [Isufi-Leus16]

⇒ M successive diffusion dynamics ⇒ AR of order M
⇒ Process is the sum of M parallel diffusions ⇒ ARMA order M
QM PM
x∞ = m=1 (I − αm S)−1 w x∞ = m=1 (I − αm S)−1 w

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 45 / 118

General linear network processes

I Combinations of all the previous are possible

xt = Hat (S)xt−1 + Hbt (S)w ⇒ xt = HAt (S)x0 + HBt (S)w

⇒ y = xt , sequential/parallel application, linear combination

⇒ Expands range of processes that can be modeled via GSP

⇒ Coefficients can change according to some control inputs
I A number of linear processes can be modeled using graph filters
⇒ Theoretical GSP results can be applied to distributed networking
⇒ Deconvolution, filtering, system id, ...
⇒ Beyond linearity possible too (more at the end of the talk)
I Links with control theory (of networks and complex systems)
⇒ Controllability, observability
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 46 / 118
Application: Explaining human learning rates

I Why do some people learn faster than others?

⇒ Can we answer this by looking at their brain activity?

I Brain activity during learning of a motor skill in 112 cortical regions

⇒ fMRI while learning a piano pattern for 20 individuals

I Pattern is repeated, reducing the time needed for execution

⇒ Learning rate = rate of decrease in execution time
brain signal
normalized
I Define a functional brain graph neighbor dista

region
temporal smo
⇒ Based on correlated activity
then cluste
I fMRI outputs a series of graph signals
time
⇒ x(t) ∈ R112 describing brain states
# entries in higher than a thres
I Does brain state variability correlate with learning? sum ( elementwise α-power in )
# changes in
# entries in higher than a thres
sum ( elementwise α-power in )
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications approximate entropy ( 47) / 118
Measuring brain state variability
I We propose three different measures capturing different time scales
⇒ Changes in micro, meso, and macro scales
I Micro: instantaneous changes higher than a threshold α
T  
X kx(t) − x(t − 1)k2
m1 (x) = 1 >α
t=1
kx(t)k2
I Meso: Cluster brain states and count the changes in clusters
T
X
m2 (x) = 1 {c(t) 6= c(t − 1)}
t=1
⇒ where c(t) is the cluster to which x(t) belongs.
I Macro: Sample entropy. Measure of complexity of time series
P P !
t s6=t 1{kx̄3 (t) − x̄3 (s)k∞ > α}
m3 (x) = − log P P
t s6=t 1{kx̄2 (t) − x̄2 (s)k∞ > α}

⇒ Where x̄r (t) = [x(t), x(t + 1), . . . , x(t + r − 1)]

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 48 / 118
Diffusion as low-pass filtering

I We diffuse each time signal x(t) across the brain graph

xdiff (t) = (I + βL)−1 x(t)

⇒ where Laplacian L = VΛV−1 and β represents the diffusion rate

I Analyzing diffusion in the frequency domain

x̃diff (t) = (I + βΛ)−1 V−1 x(t) = diag(h̃)x̃(t)

⇒ where h̃i = 1/(1 + βλi )

I Diffusion acts as low-pass filtering

Response
I High freq. components are attenuated
I β controls the level of attenuation Frequency

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 49 / 118

Computing correlation for three signals

I Variability measures consider the order of brain signal activity

I As a control, we include in our analysis a null signal time series xnull

xnull (t) = xdiff (πt )

⇒ where πt is a random permutation of the time indices

I Correlation between variability (m1 , m2 , and m3 ) and learning?

I We consider three time series of brain activity
⇒ The original fMRI data x
⇒ The filtered data xdiff
⇒ The null signal xnull

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 50 / 118

Low-pass filtering reveals correlation

I Correlation coeff. between learning rate and brain state variability

Original Filtered Null

m1 0.211 0.568 0.182
m2 0.226 0.611 0.174
m3 0.114 0.382 0.113

I Correlation is clear when the signal is filtered

⇒ Result for original signal similar to null signal
I Scatter plots for original, filtered, and null signals (m2 variability)
-3 -3 -3
9 #10 9 #10 9 #10

8 8 8

7 7 7

Learning Rate
Learning Rate

Learning Rate

6 6 6

5 5 5

4 4 4

3 3 3

2 2 2

1 1 1
0.52 0.53 0.54 0.55 0.56 0.57 0.58 0.59 0.05 0.1 0.15 0.2 0.25 0.3 0.03 0.04 0.05 0.06 0.07 0.08 0.09
Variability Variability Variability

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 51 / 118

Part II: Applications

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 52 / 118

Application domains

I Design graph filters to approximate desired network operators

I Sampling bandlimited graph signals
I Blind graph filter identification
⇒ Infer diffusion coefficients from observed output
I Network topology inference
⇒ Infer shift from collection of network diffused signals

I Many more (not covered, glad to discuss or redirect):

⇒ Statistical GSP, stationarity and spectral estimation
⇒ Filter banks
⇒ Windowing, convolution, duality...
⇒ Nonlinear GSP
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 53 / 118
Sampling bandlimited graph signals

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 54 / 118

Motivation and preliminaries
I Sampling and interpolation are cornerstone problems in classical SP
⇒ How recover a signal using only a few observations?
⇒ Need to limit the degrees of freedom: subspace, smoothness

I Graph signals: sampling thoroughly investigated

⇒ Most assume only a few values are observed
⇒ [Anis14, Chen15, Tsitsvero15, Puy15, Wang15]

I Alternative approach [Marques16, Segarra16]

⇒ GSP is well-suited for distributed networking
⇒ Incorporate local graph structure into the observation model
⇒ Recover signal using distributed local graph operators

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 55 / 118

Sampling bandlimited graph signals: Overview

I Sampling is likely to be most important inverse problem

⇒ How to find x ∈ RN using P < N observations?

I Our focus on bandlimited signals, but other models possible

⇒ x̃ = V−1 x sparse
P
⇒ x = k∈K x̃k vk , with |K| = K < N
⇒ S involved in generation of x
⇒ Agnostic to the particular form of S
I Two sampling schemes were introduced in the literature
⇒ Selection [Anis14, Chen15, Tsitsvero15, Puy15, Wang15]
⇒ Aggregation [Segarra15], [Marques15]
⇒ Hybrid scheme combining both ⇒ Space-shift sampling

I More involved, theoretical benefits, practical benefits in distr. setups

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 56 / 118

Revisiting sampling in time
I There are two ways of interpreting sampling of time signals
I We can either freeze the signal and sample values at different times

I We can fix a point (present) and sample the evolution of the signal

I Both strategies coincide for time signals but not for general graphs
⇒ Give rise to selection and aggregation sampling
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 57 / 118
Selection sampling: Definition

I Intuitive generalization to graph signals

⇒ C ∈ {0, 1}P×N (matrix P rows of IN )
⇒ Sampled signal is x̄ = Cx
I Goal: recover x based on x̄
⇒ Assume that the support of K is known (w.l.o.g. K = {k}K k=1 )
⇒ Since x̃k = 0 for k > K , define x̃K := [x̃1 , ..., x̃K ]T = ET
K x̃

I Approach: use x̄ to find x̃K , and then recover x as

x = V(EK x̃K ) = (VEK )x̃K = VK x̃K

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 58 / 118

Selection sampling: Recovery

I Number of samples P ≥ K

x̄ = Cx = CVK x̃K
⇒ (CVK ) submatrix of V

Recovery of selection sampling

If rank(CVK ) ≥ K , x can be recovered from the P values in x̄ as

x = VK x̃K = VK (CVK )† x̄

I With P = K , hard to check invertibility (by inspection)

⇒ Columns of VK (CVK )−1 are the interpolators

I In time (S = Adc ), if the samples in C are equally spaced

⇒ (CVK ) is Vandermonde (DFT) and VK (CVK )−1 are sincs
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 59 / 118
Aggregation sampling: Definition

I Idea: incorporating S to the sampling procedure

⇒ Reduces to classical sampling for time signals

I Consider shifted (aggregated) signals y(l) = Sl x

⇒ y(l) = Sy(l−1) ⇒ found sequentially with only local exchanges
(0) (1) (N−1) T
I Form yi = [yi , yi , ..., yi ] (obtained locally by node i)

I The sampled signal is

ȳi = Cyi

I Goal: recover x based on ȳi

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 60 / 118

Aggregation sampling: Recovery

I Goal: recover x based on ȳi ⇒ Same approach than before

⇒ Use ȳi to find x̃K , and then recover x as x = VK x̃K
I T
Define ūi := VK ei and recall Ψkl = λl−1
k

Recovery of aggregation sampling

Signal x can be recovered from the first K samples in ȳi as

x = VK x̃K = VK diag−1 (ūi )(CΨT EK )−1 ȳi

provided that [ūi ]k 6= 0 and all {λk }K

k=1 are distinct.

I If C = ET
K , node i can recover x with info from K − 1 hops!
⇒ Node i has to be able to capture frequencies in K
⇒ The frequencies have to distinguishable
I Bandlimited signals: Signals that can be well estimated locally
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 61 / 118
Aggregation and selection sampling: Example

I In time (S = Adc ), selection and aggregation are equivalent

⇒ Differences for a more general graph?

I Erdős-Rényi
p = 0.2, S = A,
K = 3,
non-smooth

I First 3 observations at node 4: y4 = [0.55, 1.27, 2.94]T

⇒ [y4 ]1 = x4 = −0.55, [y4 ]2 = x2 + x3 + x5 + x6 + x7 = 1.27
⇒ For this example, any node guarantees recovery
⇒ Selection sampling fails if, e.g., {1, 3, 4}

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 62 / 118

Sampling: Discussion and extensions

I Discussion on aggregation sampling

⇒ Observation matrix: diagonal times Vandermonde
⇒ Very appropriate in distributed scenarios
⇒ Different nodes will lead to different performance (soon)
⇒ Types of signals that are actually bandlimited (role of S)

I Three extensions:
⇒ Sampling in the presence of noise
⇒ Unknown frequency support
⇒ Space-shift sampling (hybrid)

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 63 / 118

Presence of noise

I Linear observation model: z̄i = CΨi x̃K + Cwi and x = VK x̃K

I BLUE interpolation (Ψi either selection or aggregation)

ˆ (i) (i) −1 −1 H H (i) −1

x̃K = [ΨH H
i C (R̄w ) CΨi ] Ψi C (R̄w ) z̄i

−1
⇒ If P = K , then x̂(i) = VK (CΨi ) z̄i
(i) (i)
I Error covariances (Re ,R̃e ) in closed form ⇒ Noise covariances?
⇒ Colored, different models: white noise in zi , in x, or in x̃K

I Metric to optimize?
i−1
(i)−1
h 
(i) (i) (i)
⇒ trace(Re ), λmax (Re ), log det(R̃e ), trace R̃e

I Select i and C to min. error ⇒ Depends on metric and noise [Marques16]

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 64 / 118

Unknown frequency support

I Falls into the class of sparse reconstruction: observation matrix?

⇒ Selec. ⇒ submatrix of unitary VK
⇒ Aggr. ⇒ Vander. × diag
[ui ]k 6= 0 and λk 6= λk 0 ⇒ full-spark

I Joint recovery and support identification (noiseless)

x̃∗ := arg min ||x̃||0

x̃
s.t. Cyi = CΨi x̃,

I If full spark ⇒ P = 2K samples suffice

⇒ Different relaxations are possible
⇒ Conditioning will depend on Ψi (e.g., how different {λk } are)

I Noisy case: sampling nodes critical

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 65 / 118
Recovery with unknown support: Example

I Erdős-Rényi
p = 0.15, 0.20, 0.25,
K = 3, non-smooth

I Three different shifts: A, (I − A) and 21 A2

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 66 / 118

Space-shift sampling

I Space-shift sampling (hybrid) ⇒ Multiple nodes and multiple shifts

Selection: 4 nodes, 1 sample Space-shift: 2 nodes, 2 samples Aggregat.: 1 node, 4 samples

I Section and aggregation sampling as particular cases

I With Ū := [diag(ū1 ), ..., diag(ūN )]T , the sampled signal is
 
z̄ = C I⊗(ΨT EK ) Ūx̃K + Cw
¯ ¯

I As before, BLUE and error covariance in close-form

I Optimizing sample selection more challenging
I More structured schemes easier: e.g., message passing
(l) (l 0 )
⇒ Node i knows yi ⇒ node i knows yj for all j ∈ Ni and l 0 < l
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 67 / 118
Sampling the US economy

I 62 economic sectors in USA + 2 artificial sectors

⇒ Graph: average flows in 2007-2010, S = A
⇒ Signal x: production in 2011
⇒ x is approximately bandlimited with K = 4

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 68 / 118

Sampling the US economy: Results

I Setup 1: we add different types of noise

⇒ Error depends on sampling node: better if more connected

I Setup 2: we try different shift-space strategies

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 69 / 118

More on sampling graph signals

I Beyond bandlimitedness
⇒ Smooth signals [Chen15]
⇒ Parsimonious in kernelized domain [Romero16]

I Strategies to select the sampling nodes

⇒ Random (sketching) [Varma15]
⇒ Optimal reconstruction [Marques16, Chepuri-Leus16]
⇒ Designed based on posterior task [Gama16]

I And more...
⇒ Low-complexity implementations [Tremblay16, Anis16]
⇒ Local implementations [Wang14, Segarra15]
⇒ Unknown spectral decomposition [Anis16]

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 70 / 118

Stationarity of random graph processes

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 71 / 118

Motivation and context

I We frequently encounter stochastic processes

I Statistical SP ⇒ tools for their understanding

I Stationarity facilitates the analysis of random signals in time

⇒ Statistical properties are time-invariant

I We seek to extend the concept of stationarity to graph processes

⇒ Network data and irregular domains motivate this
⇒ Lack of regularity leads to multiple definitions

I Classical SSP can be generalized: spectral estimation, periodograms,...

⇒ Better understanding and estimation of graph processes
⇒ Related works: [Girault 15], [Perraudin 16]
∗
Segarra, Marques, Leus, Ribeiro, Stationary Graph Processes: Nonparametric Spectral Estimation, SAM16
?
Marques, Segarra, Leus, Ribeiro, Stationary Graph Processes and Spectral Estimation, IEEE TSP (sub.)

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 72 / 118

Prelimaries: Weak stationarity in time

(1) Correlation of stationary discrete time signals is invariant to shifts

h i h i h i
Cx := E xxH = E xH (n − l)N x(n − l)N = E Sl x(Sl x)H

(2) Signal is the output of a LTI filter H excited with white noise w

x = Hw, with E wwH = I

 

(3) The covariance matrix Cx is diagonalized by the Fourier matrix

Cx = Fdiag(p)FH

The process has a power spectral density ⇒ p := diag FH Cx F

I

I Each of these definitions can be generalized to graph signals

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 73 / 118

Stationary graph processes: Shifts

Definition (shift invariance)

Process x is weakly stationary with respect to S if and only if (b > c)
h
H i h
H i
E Sa x (SH )b x = E Sa+c x (SH )b−c x

I Use a and b shifts as reference. Shift by c forward and backward

⇒ Signal is stationary if these shifts do not alter its covariance

It reduces to E xxH = E Sl x(Sl x)H when S is a directed cycle

   
I

I Time shift is orthogonal, SH = S−1 (a = 0, b = N and c = l)

I Need reference shifts because S can change energy of the signal

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 74 / 118

Stationary graph processes: Filtering

Definition (filtering of white noise)

Process x is weakly stationary with respect to S if it can be written as
the output of linear shift invariant filter H with white input w

x = Hw, with E wwH = I

 

I The filter H is linear shift invariant if ⇒ H(Sx) = S(Hx)

L
X
I Equivalently, H polynomial on the shift operator ⇒ H = hl Sl
l=0

Filter H determines color ⇒ Cx = E (Hw)(Hw)H = HHH

 
I

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 75 / 118

Stationary graph processes: Diagonalization

Definition (Simultaneous diagonalization)

Process x is weakly stationary with respect to S if the covariance Cx and
the shift S are simultaneously diagonalizable

S = V Λ VH =⇒ Cx = V diag(p) VH

I Equivalent to time definition because F diagonalizes cycle graph

The process has a power spectral density ⇒ p := diag VH Cx V

I

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 76 / 118

Equivalence of definitions and PSD

I Have introduced three equally valid definitions of weak stationarity

⇒ They are different but, pleasingly, equivalent

Proposition
Process x has shift invariant correlation matrix ⇔ it is the output of a linear
shift invariant filter ⇔ Covariance jointly diagonalizable with shift

I Shift and Filtering ⇒ How stationary signals look like (local invariance)


I Simultaneous Diagonalization ⇒ A PSD exists ⇒ p := diag VH Cx V
⇒ The PSD collects the eigenvalues of Cx and is nonnegative

Proposition
Let x be stationary in S and define the process x̃ := VH x. Then, it holds
that x̃ is uncorrelated with covariance matrix Cx̃ = E x̃x̃H = diag(p).

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 77 / 118

Weak stationary graph processes examples

Example (White noise)

I White noise w is stationary in any graph shift S = VΛVH
I Covariance Cw = σ 2 I simultaneously diagonalizable with all S

Example (Covariance matrix graphs and Precision matrices)

I Every process is stationary in the graph defined by its covariance matrix
I If S = Cx , shift S and covariance Cx diagonalized by same basis
I Process is also stationary on precision matrix S = C−1
x

Example (Heat diffusion processes and ARMA processes)

I Heat diffusion process in a graph ⇒ x = α0 (I − αL)−1 w
I Stationary in L since α0 (I − αL)−1 is a polynomial on L
I Any autoregressive moving average (ARMA) process on a graph

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 78 / 118

Power spectral density examples

Example (White noise)

Power spectral density ⇒ p = diag VH (σ 2 I)V = σ 2 1


I

Example (Covariance matrix graphs and Precision matrices)

I Power spectral density ⇒ p = diag VH (VΛVH )V = diag(Λ)

Example (Heat diffusion processes and ARMA processes)

h i
I Power spectral density ⇒ p = diag α2 (I − αΛ)−2
0

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 79 / 118

PSD estimation with correlogram & periodogram

I Given a process x, the covariance of x̃ = VH x is given by

Cx̃ := E x̃x̃H = E (VH x)(VH x)H = diag(p)

   

I Periodogram ⇒ Given samples {xr }Rr=1 , average GFTs of samples

R R
1 X 2 1 X H 2
p̂pg := |x̃r | = V xr
R r =1 R r =1

I Correlogram ⇒ Replace Cx in PSD definition by sample covariance

  X R  
  1
p̂cg := diag VH Ĉx V := diag VH xr xH
r V
R r =1

I Periodogram and correlogram lead to identical estimates p̂pg = p̂cg

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 80 / 118

Accuracy of periodogram estimator

Theorem
If the process x is Gaussian, periodogram estimates have bias and
variance
I Bias ⇒ bpg := E [p̂pg ] − p = 0
2 2
 
I Variance ⇒ Σpg := E (p̂pg − p)(p̂pg − p)H = R diag (p)

I The periodogram is unbiased but the variance is not too good

⇒ Quadratic in p. Same as time processes

I Alternative nonparametric methods to reduce variance

⇒ Average windowed periodogram
⇒ Filterbanks
⇒ Bias - variance tradeoff characterized [Marques16,Segarra16]

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 81 / 118

Parametric PSD estimation

I PG and CG are examples of non-parametric estimators

I Parametric ARMA estimation

PL
⇒ Model x = l=0 hl Sl w with w white
⇒ PSD is px (h) = |Ψh|2
⇒ Given xr , compute p̂pg and find ĥ = arg minh d(p̂pg , px (h))
⇒ Set p̂MA = px (ĥ) = |Ψĥ|2
⇒ General estimation problem nonconvex
⇒ Particular cases (S  0 and h  0) tractable

I Other parametric models (sum of frequency basis) possible too

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 82 / 118

Average periodogram

I MSE of periodogram as a function of the nr. of observations R

I Baseline ER random graph (N = 100 and p = 0.05) and S = A

I Observe filtered white Gaussian noise and estimate PSD
Normalized MSE

2
Baseline ER
Smaller ER
1 Larger PSD
Small-world

0
1 3 5 7 9 11 13 15 17 19
Number of observations

I Normalized MSE evolves as 2/R as expected

⇒ Invariant to size, topology, and PSD

I Same behavior observed in non-Gaussian processes (theory not valid)

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 83 / 118
Windowed average periodogram

I Performance of local windows and random windows

I Block stochastic graph (N = 100, 10 communities) and small world
I Process filters white noise with different number of taps
2.5
Bias squared
Variance 2 Local window (L=10)
Theoretical error Local window (L=2)
2 Empirical error Random window (L=10)
Random window (L=2)

Normalized MSE
1.5
Normalized MSE

1.5

1
1

0.5
0.5

0
0 1 3 5 7 9 11 13 15 17 19
Single window Ten local wind. Ten random wind. Number of windows

I The use of windows introduces bias but reduces total error (MSE)
I Local windows work better than random windows
⇒ Advantage of local windows is larger for local processes
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 84 / 118
Opinion source identification

I Opinion diffusion in Zachary’s karate club network (N = 34)

I Observed opinion x obtained by diffusing sparse white rumor w
I Given {xr }Rr=1 generated from unknown {wr }Rr=1
⇒ Diffused through filter of unknown nonnegative coefficients β
I Goal ⇒ Identify the support of each rumor wr
I First ⇒ Estimate β from Moving Average PSD estimation
I Second ⇒ Solve R sparse linear regressions to recover supp(wr )

0.45
< =0
0.4 < = 0.10
Source identification error

< = 0.15
0.35 < = 0.20

0.3

0.25

0.2

0.15

0.1

0.05
10 15 20 25 30
Number of observations

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 85 / 118

PSD of face images

I PSD estimation for spectral signatures of faces of different people

I 100 grayscale face images {xi }100
i=1 ∈ R
10304
(10 images × 10 people)

I Consider xi as realization graph process that is Stationary on Ĉx

(j)
I Construct Ĉx = V(j) Λ(j)
c V
H(j)
based on images of person j
#105
3.5
5 8 Individual 1
Individual 2
10 3 Individual 3
6
15
Frequency index

Normalized MSE
4 2.5
20
25 2 2
30
0
35 1.5
-2
40
1
45 -4
50 0.5
10 20 30 40 50 0 5 10 15 20
Frequency index Number of windows

I Process of person j approximately stationary in Ĉx (left)

I Use windowed average periodogram to estimate PSD of new face
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 86 / 118
Stationarity: Takeaways

I Extended the notion of weak stationarity for graph processes

I Three definitions inspired in stationary time processes
⇒ Shown all of them to be equivalent

I Defined power spectral density and studied its estimation

I Generalized classical non-parametric estimation methods
⇒ Periodogram and correlogram where shown to be equivalent
⇒ Windowed average periodogram, filter banks
I Generalized classical ARMA parametric estimation methods
⇒ Particular cases tractable

I Extensions
⇒ Other parametric schemes
⇒ Space-time variation

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 87 / 118

Network topology inference

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 88 / 118

Motivation and context

I Network topology inference from nodal observations [Kolaczyk’09]

⇒ Approaches use Pearson correlations to construct graphs [Brovelli04]
⇒ Partial correlations and conditional dependence [Friedman08, Karanikolas16]

I Key in neuroscience [Sporns’10]

⇒ Functional net inferred from activity
I Most GSP works assume that S (hence the graph) is known
⇒ Analyze how the characteristics of S affect signals and filters

I We take the reverse path

⇒ How to use GSP to infer the graph topology?
⇒ [Dong15, Mei15, Pavez16, Pasdeloup16]
†Segarra, Marques, Mateos, Ribeiro, Network Topology Identification from Spectral Templates, IEEE SSP16
‡Segarra, Marques, Mateos, Ribeiro, Network Topology Inference from Spectral Templates, IEEE TSP (sub.)

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 89 / 118

Structure from graph stationarity

I Given a set of signals {xr }Rr=1 find S

⇒ We view signals as samples of random graph process x
⇒ AS. x is stationary in S

I Equivalent to “x is the linear diffusion of a white input”

∞
Y ∞
X
x = α0 (I − αl S)w = βl Sl w
l=1 l=0

⇒ Examples: Heat diffusion, structural equation models

x = (I − αL)−1 w x = Ax + w

I We say the graph shift S explains the structure of signal x

I Key point after assuming stationarity: eigenvectors of the covariance

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 90 / 118

Covariance and shift: Eigenvectors

I The covariance matrix of the stationary signal x = Hw is

Cx = E xxH = HE wwH HH = HHH

  


⇒ Since H is diagonalized by V, so is the covariance Cx

PL−1 2
Cx = V l=0 hl Λl VH = V diag(p) VH

I Any shift with eigenvectors V can explain x

⇒ G and its specific eigenvalues have been obscured by diffusion
Observations
(a) There are many shifts that can explain a signal x
(b) Identifying the shift S is just a matter of identifying the eigenvalues
(c) In correlation methods the eigenvalues are kept unchanged
(d) In precision methods the eigenvalues are inverted

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 91 / 118

Our approach for topology inference

I We propose a two-step approach for graph topology identification

Inferred"
Input"
network"
A"priori"info,"desired"
topological"features"

STEP%1:% STEP%2:%
Inferred" Iden-fy%eigenvalues%to%
Iden-fy%the%eigenvectors% eigenvectors"
of%the%shi9% obtain%a%suitable%shi9%

I Beyond diffusion ⇒ alternative sources for spectral templates V

⇒ Graph sparsification, network deconvolution,...

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 92 / 118

STEP 1: Other sources of spectral templates

1) Graph sparsification
I Goal: given Sf find sparser S with same eigenvectors
⇒ Find Sf = Vf Λf VfH and set V = Vf
⇒ Otentimes referred to as network deconvolution problem
2) Nodal relation assumed by a given transform
I GSP: decompose S = VΛVH and set VH as GFT
I SP: some transforms T known to work well on specific data
I Goal: given T, set VH = T and identify S ⇒ intuition on data relation

DCTs: i–iii

3) Implementation of linear network operators

I Goal: distributed implementation of linear operator B via graph filter
⇒ Feasible if B and S share eigenvectors ⇒ Like 1) with Sf = B

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 93 / 118

STEP 2: Obtaining the eigenvalues

I Given V, there are many possible S = Vdiag(λ)VH

⇒ We can use extra knowledge/assumptions to choose one graph
⇒ Of all graphs, select one that is optimal in some sense
N
X
S∗ := argmin f (S, λ) s. to S = λk vk vkH , S ∈ S (1)
S,λ
k=1

I Set S contains all admissible scaled adjacency matrices

S := {S | Sij ≥ 0, S ∈ MN, Sii = 0,

P
j S1j = 1}

⇒ Can accommodate Laplacian matrices as well

I Problem is convex if we select a convex objective f (S, λ)

⇒ Minimum energy (f (S) = kSkF ), Fast mixing (f (λ) = −λ2 )

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 94 / 118

Size of the feasibility set

I The feasibility set in (1) is generally small ⇒ Why?

⇒ We search over λ ∈ RN , we have N linear constraints Sii = 0

I This helps in the optimization, to be rigorous

⇒ Define W := V V where is the Khatri-Rao product
⇒ Denote by D the index set such that vec(S)D = diag(S)

Assume that (1) is feasible, then it holds that rank(WD ) ≤ N − 1.

If rank(WD ) = N − 1, then the feasible set of (1) is a singleton.

I Convex feasibility set ⇒ Search for the optimal solution may be easy

I Simulations will show that rank(WD ) = N −1 arises in practice

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 95 / 118

Sparse recovery

I Whenever the feasibility set of (1) is non-trivial

⇒ f (S, λ) determines the features of the recovered graph

Ex: Identify the sparsest shift S∗0 that explains observed signal structure
⇒ Set the cost f (S, λ) = kSk0
N
X
S∗0 = argmin kSk0 s. to S = λk vk vkT , S ∈ S
S,λ
k=1

I Problem is not convex, but can relax to `1 norm minimization

N
X
S∗1 := argmin kSk1 s. to S = λk vk vkH , S ∈ S
S,λ
k=1

I Does the solution S∗1 coincide with the `0 solution S∗0 ?

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 96 / 118

Recovery guarantee

†
I Denoting by mT
i the i-th row of M := (I − WW )D c
⇒ Construct R := [m2 −m1 , . . . , mN−1 −m1 , mN , . . . , m|Dc | ]T
⇒ Denote by K the indices of the support of s∗0 = vec(S∗0 )

S∗1 and S∗0 coincide if the two following conditions are satisfied:
1) rank(RK ) = |K|; and
2) There exists a constant δ > 0 such that

ψR := kIKc (δ −2 RRT + IT −1 T
Kc IKc ) IK k∞ < 1.

I Cond. 1) ensures uniqueness of solution S∗1

I Cond. 2) guarantees existence of a dual certificate for `0 optimality

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 97 / 118

Noisy and incomplete spectral templates

I We might have access to V̂, a noisy version of the spectral templates

⇒ With d(·, ·) denoting a (convex) distance between matrices

PN
min kSk1 s. to Ŝ = k=1 λk v̂k v̂k T , S ∈ S, d(S, Ŝ) ≤ 
{S,λ,Ŝ}

I Recovery result similar to the noiseless case can be derived

⇒ Conditions under which we are guaranteed d(S∗ , S∗0 ) ≤ C 

I Partial access to V ⇒ Only K known eigenvectors [v1 , . . . , vK ]

PK T
min kSk1 s. to S = SK̄ + k=1 λk vk vk , S ∈ S, SK̄ vk = 0
{S,SK̄ ,λ}

I Incomplete and noisy scenarios can be combined

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 98 / 118

Topology inference in random graphs

I Erdős-Rényi graphs of varying size N ∈ {10, 20, . . . , 50}

⇒ Edge probabilities p ∈ {0.1, 0.2, . . . , 0.9}
I Recovery rates for adjacency (left) and normalized Laplacian (mid)
1 60
All realizations
10 10 Successful recovery
50 Unique solution
0.8
20 20 40

Frequency
0.6
30 30

N
N

30
0.4 40 40 20

0.2 50 50 10

0 0.1 0.3 0.5 0.7 0.9 0.1 0.3 0.5 0.7 0.9 0
2 2.5 5 6 7 8 9
p p Rank

I Recovery is easier for intermediate values of p

I Rate of recovery related to the rank(WD ) (histogram N = 10,p = 0.2)
⇒ When rank is N − 1, recovery is guaranteed
⇒ As rank decreases, there is a detrimental effect on recovery

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 99 / 118

Sparse recovery guarantee

I Generate 1000 ER random graphs (N = 20, p = 0.1) such that

⇒ Feasible set is not a singleton
⇒ Cond. 1) in sparse recovery theorem is satisfied

I Noiseless case: `1 norm guarantees recovery as long as ψR < 1

180
Recovery Success
160 Recovery Failure
140
120
100
80
60
40
20
0
0 1 2 3 4 5 6 7

I Condition is sufficient but not necessary

⇒ Tightest possible bound on this matrix norm

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 100 / 118
Inferring brain graphs from noisy templates

I Identification of structural brain graphs N = 66

I Test recovery for noisy spectral templates V̂
⇒ Obtained from sample covariances of diffused signals
0.3
Patient 1
0.25 Patient 2
Patient 3
Recovery error 0.2

0.15

0.1

0.05

0
10 3 10 4 10 5 10 6
Number of Observations

I Recovery error decreases with increasing number of observed signals

⇒ More reliable estimate of the covariance ⇒ Less noisy V̂
I Brain of patient 1 is consistently the hardest to identify
⇒ Robustness for identification in noisy scenarios
I Traditional methods like graphical lasso fail to recover S
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 101 / 118
Inferring social graphs from incomplete templates

I Identification of multiple social networks N = 32

⇒ Defined on the same node set of students from Ljubljana
I Test recovery for incomplete spectral templates V̂ = [v1 , . . . , vK ]
⇒ Obtained from a low-pass diffusion process
⇒ Repeated eigenvalues in Cx introduce rotation ambiguity in V
1
Network 1
Network 2
0.8 Network 3
Network 4
Recovery error

0.6

0.4

0.2

0
16 18 20 22 24 26 28
Number of spectral templates

I Recovery error decreases with increasing nr. of spectral templates

⇒ Performance improvement is sharp and precipitous

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 102 / 118
Performance comparisons

I Comparison with graphical lasso and sparse correlation methods

I Evaluated on 100 realizations of ER graphs with N = 20 and p = 0.2

0.9
0.8
F-measure

0.7 Our for H1

Our for H
0.6 2
Correl. for H 1
0.5 Correl for H2
GLasso for H
0.4 1
GLasso for H 2
0.3

0.2
10 1 10 2 10 3 10 4 10 5 10 6
Number of observations

I Graphical lasso implicitly assumes a filter H1 = (ρI + S)−1/2

⇒ For this filter spectral templates work, but not as well (MLE)

I For general diffusion filters H2 spectral templates still work fine

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 103 / 118
Inferring direct relations

I Our method can be used to sparsify a given network

I Keep direct and important edges or relations
⇒ Discard indirect relations that can be explained by direct ones
I Use eigenvectors V̂ of given network as noisy templates
I Infer contact between amino-acid residues in BPT1 BOVIN
⇒ Use mutual information of amino-acid covariation as input

5 5 5 5

10 10 10 10

15 15 15 15

20 20 20 20

25 25 25 25

30 30 30 30

35 35 35 35

40 40 40 40

45 45 45 45

50 50 50 50

10 20 30 40 50 10 20 30 40 50 10 20 30 40 50 10 20 30 40 50

Ground truth Mutual info. Network deconv. Our approach

I Network deconvolution assumes a specific filter model [Feizi13]

⇒ We achieve better performance by being agnostic to this
Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 104 / 118
Topology ID: Takeaways

I Network topology inference cornerstone problem in Network Science

I Most GSP works analyze how S affect signals and filters
I Here, reverse path: How to use GSP to infer the graph topology?

I Our GSP approach to network topology inference

⇒ Two step approach: i) Obtain V; ii) Estimate S given V

I How to obtain the spectral templates V

⇒ Based on covariance of diffused signals
⇒ Other sources too: net operators, data transforms

I Infer S via convex optimization

⇒ Objectives promotes desirable properties
⇒ Constraints encode structure a priori info and structure
⇒ Formulations for perfect and imperfect templates
⇒ Sparse recovery results for both adjacency and Laplacian

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 105 / 118
Wrapping up

Motivation and preliminaries

Part I: Fundamentals
Graph signals and the shift operator
Graph Fourier Transform (GFT)
Graph filters and network processes

Part II: Applications

Sampling graph signals
Stationarity of graph processes
Network topology inference

Concluding remarks

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 106 / 118
Concluding remarks

I Network science and big data pose new challenges

⇒ GSP can contribute to solve some of those challenges
⇒ Well suited for network (diffusion) processes

I Central elements in GSP: graph-shift operator and Fourier transform

I Graph filters: operate graph signals

⇒ Polynomials of the shift operator that can be implemented locally

I Network diffusion/percolations processes via graph filters

⇒ Successive/parallel combination of local linear dynamics
⇒ Possibly time-varying diffusion coefficients
⇒ Accurate to model certain setups
⇒ GSP yields insights on how those processes behave

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 107 / 118
Concluding remarks

I GSP results can be applied to solve practical problems

⇒ Sampling, interpolation (network control)
⇒ Input and system ID (rumor ID)
⇒ Shift design (network topology ID)

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 108 / 118
Looking ahead

I First step to challenging problems: social nets, brain signals

I Motivates further research:

⇒ Space-time variation
⇒ Changing topologies
⇒ Nonlinear approaches
⇒ Local, reduced-complexity algorithms

I Thanks!
⇒ If you have questions, feel free to contact me by e-mail
[email protected] or any of the other authors.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 109 / 118
References

We include a list of our published work in graph signal processing (GSP)

categorized by topic. We also include relevant works by other authors. This
latter list is not intended to be exhaustive but rather its purpose is to guide the
interested reader to pertinent publications in different areas of graph signal
processing.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 110 / 118
References: our work

Sampling bandlimited graph signals

A. G. Marques, S. Segarra, G. Leus, and A. Ribeiro, ”Sampling of Graph Signals with Successive
Local Aggregations”, IEEE Trans. on Signal Process., vol. 64, no. 7, pp. 1832 - 1843, Apr. 2016.
S. Segarra, A. G. Marques, G. Leus and A. Ribeiro, ”Space-shift sampling of graph signals”, Proc.
of IEEE Intl. Conf. on Acoustics, Speech and Signal Process., Shanghai, China, March 20-25,
2016.
S. Segarra, A. G. Marques, G. Leus and A. Ribeiro, ”Aggregation Sampling of Graph Signals in the
Presence of Noise”, Proc. of IEEE Intl. Wrksp. on Computational Advances in Multi-Sensor
Adaptive Processing, Cancun, Mexico, Dec. 13-16, 2015.
S. Segarra, A. G. Marques, G. Leus, and A. Ribeiro, ”Sampling of Graph Signals: Successive Local
Aggregations at a Single Node”, Proc. of 49th Asilomar Conf. on Signals, Systems, and
Computers, Pacific Grove, CA, Nov. 8-11, 2015.
F. Gama, A. G. Marques, G. Mateos, and A. Ribeiro, ”Rethinking Sketching as Sampling: Efficient
Approximate Solution to Linear Inverse Problems”, Proc. of IEEE of Global Conf. on Signal and
Info. Process., Washington DC, Dec. 7-9, 2016.
F. Gama, A. G. Marques, G. Mateos, and A. Ribeiro, ”Rethinking Sketching as Sampling: Linear
Transforms of Graph Signals”, Proc. of 50th Asilomar Conf. on Signals, Systems, and Computers,
Pacific Grove, CA, Nov. 6-9, 2016.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 111 / 118
References: our work

Interpolating graph signals

S. Segarra, A. G. Marques, G. Leus, and A. Ribeiro, ”Reconstruction of Graph Signals: Percolation
from a Single Seeding Node”, Proc. of IEEE of Global Conf. on Signal and Info. Process.,
Orlando, FL, Dec. 14-16, 2015.
S. Segarra, A. G. Marques, G. Leus, and A. Ribeiro, ”Interpolation of Graph Signals Using
Shift-Invariant Graph Filters”, Proc. of European Signal Process. Conf., Nice, France, Aug.
31-Sep. 4, 2015.
S. Segarra, A. G. Marques, G. Leus, and A. Ribeiro, ”Reconstruction of Graph Signals through
Percolation from Seeding Nodes”, IEEE Trans. on Signal Process., vol. 64, no. 16, pp. 4363
4378, Aug. 2016.

Graph filter design and network operators

S. Segarra, A. G. Marques, and A. Ribeiro, ”Distributed Implementation of Network Linear
Operators using Graph Filters”, Proc. of 53rd Allerton Conf. on Commun. Control and
Computing, Univ. of Illinois at U-C, Monticello, IL, Sept. 30- Oct. 2, 2015.
S. Segarra, A. G. Marques, and A. Ribeiro, ”Distributed Network Linear Operators using Node
Variant Graph Filters”, Proc. of IEEE Intl. Conf. on Acoustics, Speech and Signal Process.,
Shanghai, China, March 20-25, 2016.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 112 / 118
References: our work

Blind graph deconvolution

S. Segarra, G. Mateos, A. G. Marques, and A. Ribeiro, ”Blind Identification of Graph Filters with
Sparse Inputs: Unknown support”, Proc. of IEEE Intl. Conf. on Acoustics, Speech and Signal
Process., Shanghai, China, March 20-25, 2016.
S. Segarra, G. Mateos, A. G. Marques, and A. Ribeiro, ”Blind Identification of Graph Filters with
Sparse Inputs”, Proc. of IEEE Intl. Wrksp. on Computational Advances in Multi-Sensor Adaptive
Processing, Cancun, Mexico, Dec. 13-16, 2015.
S. Segarra, G. Mateos, A. G. Marques, and A. Ribeiro, ”Blind Identification of Graph Filters”,
IEEE Trans. Signal Process. (arXiv:1604.07234 [cs.IT])

GSP-based network topology inference

S. Segarra, A. G. Marques, G. Mateos, and A. Ribeiro, ”Network Topology Identification from
Imperfect Spectral Templates”, Proc. of 50th Asilomar Conf. on Signals, Systems, and
Computers, Pacific Grove, CA, Nov. 6-9, 2016.
S. Segarra, A. G. Marques, G. Mateos, and A. Ribeiro, ”Network Topology Identification from
Spectral Templates”, Proc. of IEEE Intl. Wrksp. on Statistical Signal Process., Palma de
Mallorca, Spain, June 26-29, 2016.
S. Segarra, A. G. Marques, G. Mateos, and A. Ribeiro, ”Network Topology Inference from Spectral
Templates”, IEEE Trans. Signal Process., (arXiv:1608.03008 [cs.SI]).

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 113 / 118
References: our work

Stationary graph processes

A. G. Marques, S. Segarra, G. Leus, and A. Ribeiro, ”Stationary Graph Processes and Spectral
Estimation”, IEEE Trans. Signal Process. (arXiv:1603.04667 [cs.SY])
S. Segarra, A. G. Marques, G. Leus, and A. Ribeiro, ”Stationary Graph Processes: Nonparametric
Power Spectral Estimation ”, Proc. of IEEE Sensor Array and Multichannel Signal Process.
Wrksp., Rio de Janeiro, Brazil, July 10-13, 2016.

Median graph filters

S. Segarra, A. Marques, G. Arce, and Alejandro Ribeiro, ”Center-weighted Median Graph Filters”,
Proc. of IEEE of Global Conf. on Signal and Info. Process., Washington DC, Dec. 7-9, 2016.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 114 / 118
References: GSP

General references
D. Shuman, S. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, The emerging field of signal
processing on graphs: Extending highdimensional data analysis to networks and other irregular
domains, IEEE Signal Process. Mag., vol. 30, no. 3, pp. 83-98, Mar. 2013.
A. Sandryhaila and J. Moura, Discrete signal processing on graphs, IEEE Trans. Signal Process.,
vol. 61, no. 7, pp. 1644-1656, Apr. 2013.
A. Sandryhaila and J. Moura, Discrete signal processing on graphs: Frequency analysis, IEEE
Trans. Signal Process., vol. 62, no. 12, pp. 3042-3054, June 2014.
A. Sandryhaila and J. M. F. Moura, Big Data analysis with signal processing on graphs, IEEE
Signal Process. Mag., vol. 31, no. 5, pp. 80-90, 2014.
M. Rabbat and V. Gripon, Towards a spectral characterization of signals supported on small-world
networks, in IEEE Intl. Conf. Acoust., Speech and Signal Process. (ICASSP), May 2014, pp.
4793-4797.
A. Agaskar, Y.M. Lu, A spectral graph uncertainty principle, IEEE Trans. Info. Theory, vol. 59,
no. 7, pp. 4338-4356, 2013.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 115 / 118
References: GSP

Filtering
D. I. Shuman, P. Vandergheynst, and P. Frossard, Distributed signal processing via Chebyshev
polynomial approximation, CoRR, vol. abs/1111.5239, 2011.
S. Safavi and U. Khan, Revisiting finite-time distributed algorithms via successive nulling of
eigenvalues, IEEE Signal Process. Lett., vol. 22, no. 1, pp. 54-57, Jan. 2015.

Sampling
A. Anis, A. Gadde, and A. Ortega, Towards a sampling theorem for signals on arbitrary graphs, in
IEEE Intl. Conf. Acoust., Speech and Signal Process. (ICASSP), May 2014, pp. 3864-3868.
I. Shomorony and A. S. Avestimehr, Sampling large data on graphs, arXiv preprint
arXiv:1411.3017, 2014.
S. Chen, R. Varma, A. Sandryhaila, and J. Kovacevic, Discrete signal processing on graphs:
Sampling theory, IEEE. Trans. Signal Process., vol. 63, no. 24, pp. 6510-6523, Dec. 2015.
M. Tsitsvero, Mikhail, S. Barbarossa, and P. Di Lorenzo. Signals on graphs: Uncertainty principle
and sampling, arXiv preprint arXiv:1507.08822, 2015.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 116 / 118
References: GSP

Interpolation and reconstruction

S. Narang, A. Gadde, and A. Ortega, Signal processing techniques for interpolation in graph
structured data, in IEEE Intl. Conf. Acoust., Speech and Signal Process. (ICASSP), May 2013,
pp. 5445-5449.
S. Narang, A. Gadde, E. Sanou, and A. Ortega, Localized iterative methods for interpolation in
graph structured data, in Global Conf. on Signal and Info. Process. (GlobalSIP), Dec. 2013, pp.
491-494.
S. Chen, A. Sandryhaila, J. M. Moura, and J. Kovacevic, Signal recovery on graphs: Variation on
Graphs, IEEE Trans. Signal Process., vol. 63, no. 17, pp. 4609-4624, Sep. 2015.
X. Wang, P. Liu, and Y. Gu, Local-set-based graph signal reconstruction, IEEE Trans. Signal
Process., vol. 63, no. 9, pp. 2432-2444, Sept. 2015.
X. Wang, M. Wang, and Y. Gu, A distributed tracking algorithm for reconstruction of graph
signals, IEEE J. Sel. Topics Signal Process., vol. 9, no. 4, pp. 728-740, June 2015.
D. Romero, M. Meng, and G. Giannakis. Kernel-based Reconstruction of Graph Signals, arXiv
preprint arXiv:1605.07174, 2016.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 117 / 118
References: GSP

Topology inference
V. Kalofolias, How to learn a graph from smooth signals, arXiv preprint arXiv:1601.02513, 2016.
X. Dong, D. Thanou, P. Frossard, and P. Vandergheynst, Learning laplacian matrix in smooth
graph signal representations, arXiv preprint arXiv:1406.7842, 2014.
B. Pasdeloup, V. Gripon, G. Mercier, D. Pastor, M. Rabbat, Characterization and inference of
weighted graph topologies from observations of diffused signals, arXiv preprint arXiv:1605.02569,
2016.
J. Mei and J. Moura, Signal processing on graphs: Estimating the structure of a graph, in IEEE
Intl. Conf. Acoust., Speech and Signal Process. (ICASSP), 2015, pp. 54955499.
E. Pavez and A. Ortega, Generalized Laplacian precision matrix esti- mation for graph signal
processing, in IEEE Intl. Conf. Acoust., Speech and Signal Process. (ICASSP), Shanghai, China,
Mar. 20-25, 2016.

Stationarity
N. Perraudin and P. Vandergheynst, Stationary signal processing on graphs, arXiv preprint
arXiv:1601.02522, 2016.
B. Girault, Stationary graph signals using an isometric graph translation,” Proc. IEEE European
Signal Process. Conf. (EUSIPCO), 2015.

Marques, Segarra, Ribeiro Graph SP: Fundamentals and Applications 118 / 118