Séquence n°10

“Travailler avec des données structurées :

Graph Neural Network (GNN)”
 Course materials (pdf)

Practical work environment

https://fidle.cnrs.fr
Powered by CNRS CRIC, Corrected notebooks
and UGA DGDSI of Grenoble, thanks !

Videos (YouTube)

Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) (*) Procedure via Docket or pip
https://creativecommons.org/licenses/by-nc-nd/4.0/ Remember to get the latest version !
Questions and answers :
https://fidle.cnrs.fr/q2a
Accompanied by:
IA Support (dream) Team of IDRIS
Directed by:
Agathe, Baptiste et Yanis - UGA/DAPI
Léo, Kamel, Thibaut - IDRIS
https://fidle.cnrs.fr/listeinfo
Fidle information list
http://fidle.cnrs.fr/agoria
AI exchange list
 https://listes.services.cnrs.fr/wws/info/devlog
List of ESR* « Software developers » group

https://listes.math.cnrs.fr/wws/info/calcul
List of ESR* « Calcul » group

(*) ESR is Enseignement Supérieur et Recherche, french

universities and public academic research organizations
Abonnez-vous !
https ://fidle.cnrs.fr/youtube
https://www.youtube.com/@CNRS-FIDLE
 Le magazine de actualité en IA

Vendredi 16 février, 10h00

http://www.idris.fr/panoramia.html
https://www.youtube.com/@IDRISCNRS
 Envoyez-nous
des photos
« cartes postales ! »

Hakone, Japon
From : Pauline K.
Roadmap

Graphs are everywhere

➔ Complex data structures
➔ Basics of graph theory

Learning on Graphs
➔ Graph embedding
➔ Transductive and inductive learning
➔ Tasks on graph learning

A few examples
➔ Taxonomy of methods
➔ Graph convolution
➔ Message passing
➔ Graph Transformer
Roadmap

Graphs are everywhere

➔ Complex data structures
➔ Basics of graph theory

Learning on Graphs
➔ Graph embedding
➔ Transductive and inductive learning
➔ Tasks on graph learning

A few examples
➔ Taxonomy of methods
➔ Graph convolution
➔ Message passing
➔ Graph Transformer
Data structures: Euclid and Text

Highly ordered data

Rebirth of Deep learning was thanks to pictures, text and speech recognition

i-N

Neighborhood: i-1 i i+1

i+N
12
 Data structures: Data is not always euclidean
LIDAR Molecules

Complex geometries

Meshes Geometric deep learning

7
45

1
Neighborhood:
8
29
42

13
Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges https://arxiv.org/abs/2104.13478
Graphs are everywhere

Data as a set of interconnected entities

14
Graphs are everywhere
Social networks Molecules Capillary networks Particle physics
1
Denis

??

?? Bob

Charly

Alice

Directions recommendation Knowledge graphs Many other fields

– Biology
Living
Things
– Recommendation systems
is
is
– Computer vision
Plants
Animals
– Medical diagnosis
is is is
3 eat
– Robotics
Dogs Cows Herbs
– ...

[1] Erbertseder, K., Reichold, J., Flemisch, B., Jenny, P., & Helmig, R. (2012). A coupled discrete/continuum model for describing cancer-therapeutic transport in the lung. PLoS One, 7(3), e31966.
[2] J. Shlomi, P. Battaglia, and J.-R. Vlimant, “Graph neural networks in particle physics,” Mach. Learn.: Sci. Technol., vol. 2, no. 2, p. 021001, Jan. 2021, doi: 10.1088/2632-2153/abbf9a.
[3] A. Derrow-Pinion et al., “ETA Prediction with Graph Neural Networks in Google Maps,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management 15
New York, NY, USA, Oct. 2021, pp. 3767–3776. doi: 10.1145/3459637.3481916.
Vocabulary
Graph
Node
Node
g e
Ed

ge
Ed

Edge
Node
Node Edge

ge
Ed
Node
Node

16
Node/vertex Some example of nodes

Persons Atoms Intersections Particles

Denis

??

?? Bob

Charly

Alice

Road sections Concepts Many other fields

– Biology: An aminoacid in a protein
Living
Things
– Recommendation systems: A customer
is
is
– Computer vision: An object in a picture
Plants
Animals
– Medical diagnosis: Brain region (MRI)
is is is

eat
– Robotics: Joints
Dogs Cows Herbs
– ...

17
Edge Some example of edges

Relationship Type of bond Vessel Decayed to

Denis

??

?? Bob

Charly

Alice

Time Statement Many other fields

– Biology: Distance between residues
Living
Things
– Recommendation systems: Connected
is
is customers
Plants
Animals – Computer vision: Interaction between objects
is is is – Medical diagnosis: Interaction between brain
eat region (MRI)
Dogs Cows Herbs

– Robotics: connection between joints

– ...
18
Edge: orientation
A relationship can be symmetrical or not between nodes
Undirected graphs Directed graphs
Anneke van
H O C Giersbergen

H C C X
H N C N
C
C C N TV
O N
H C H
H Alterna2 http://www.alterna2.com, CC BY 2.0 <https://creativecommons.org/licenses/by/2.0>, via Wikimedia Commons
19
Edge: weight

Edges can carry information → edge weight

20
Graphs store information: Features
Graphs can store information on nodes, edges and globally

Globally Nodes Edges

Social Network Group of interest,... Name, age, job,... Is friend, follows, family,...
Molecule Is a drug, energy,... Atomic number,... Bond order,...
Citations Field,... Article,... Was cited,...
Particle physics Experiment,... Particle,... Decayed to,...
Motion capture Character,... Joints,... Is connected to,...
Natural language Paragraph,... Group of words,... Refers to,...

It can be a number, a concept, ...

21
Formal definition
G = (V, E): a set of nodes and edges
Features
V E G
{y } {y } {y }
i i i

{v i }i∈V {ei }i ∈E

22
Question break
Graph: Complexity

Number of neighbors

● The inner structure of a graph can vary a lot

● The number of edges/nodes might vary a lot from one graph to another
● One single graph can contain several thousand of nodes/edges
...
24
●
Graph: Paths
A path is a sequence of edges connecting 2 nodes

Undirected graph Directed graph

cycle

25
Graph: Node proximity and centrality

Node centrality
Measure how many paths goes through the node

Node proximity
● 1st order: w between node i and j
i,j
● 2nd order: similarity of neighborhood structure

● Higher orders possible

26
Question break
Graph representation

1000110110

?
28
Graph representation

Random numbering of nodes

29
Graph representation
{
Adjacency matrix W (i , j)= w i , j if there is an edge
0 if not

Undirected Symmetric

Directed

30
Graph representation

Adjacency list

Adjacency list: [[5, 4],

[8, 1],
[4, 8], [4,3],
[3, 4],
[1, 7], [1, 2],
[2, 4], [2, 6],
[7,1],
[6, 2],
[9, 2]]
Edges: [0.4, 0.4, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
0.4, 1.0, 1.0, 1.0]

31
Graph representation

Adjacency list: [[5, 4],

[8, 1],
[4, 8], [4,3],
[3, 4],
[1, 7], [1, 2],
[2, 4], [2, 6],
[7,1],
[6, 2],
[9, 2]]

Edges: [0.4, 0.4, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,

0.4, 1.0, 1.0, 1.0]

● Scale V2 → lot of space ● Scale E → less space

● Might be sparse ● Might be difficult to find an edge
● Easy to find an edge

V = number of nodes/vertices
E = number of edges

32
https://www.geeksforgeeks.org/comparison-between-adjacency-list-and-adjacency-matrix-representation-of-graph/
Graph representation
● Edge weights are stored either directly in the adjacency matrix, or in an independent tensor.

Adjacency matrix Adjacency list: [[5, 4],

1.0 [8, 1],
1.2 [4, 8], [4,3],
[3, 4],
3.7 8.0 [1, 7], [1, 2],
4.0 [2, 4], [2, 6],
5.0 1.0 [7,1],
1.0 3.0 [6, 2],
4.2 [9, 2]]
1.0 Edges: [0.4, 1.4, 2.4, 9.0, 1.0, 5.0, 1.7, 3.0,
7.0 0.4, 1.3, 7.0, 6.2]

● Information (features) on nodes and graphs will also be stored in independent tensors.

Nodes: [4.1, 4.2, 6.4, 1.0, 1.0, 5.0, 1.7,

3.0, 5.0]
Graph: [8.0]
33
Useful Matrices

Adjacency W Weight of edges

Diagonal matrix with number

Degree D
of edges for each node

Laplacian L D-W

Node Features X Information stored

34
Question break
Roadmap

Graphs are everywhere

➔ Complex data structures
➔ Basics of graph theory

Learning on Graphs
➔ Graph embedding
➔ Transductive and inductive learning
➔ Tasks on graph learning

A few examples
➔ Taxonomy of methods
➔ Graph convolution
➔ Message passing
➔ Graph Transformer
Graph embedding

● We need to find a representation of the graph that is processable

37
Graph embedding
 Features stored in nodes/edges/graphs are not
n2 easily processed.
n1 n1

n2

We transform the features into a vector in
n3
n4 the latent space (Dimension is a
n3 hyperparameter).
n4  The embedding has to be suited for the task →
Learnable.

n2 n2

n1 n1
n4
n3

n3
n4

38
Transductive learning
The model has access to the complete graph
It is not possible to add new nodes
Node labeling

39
Transductive learning
The model has access to the complete graph
It is not possible to add new nodes
Find new edges

40
Inductive learning
● The model has access only to a part of the graph (train set)
● Adding new nodes is possible

● Generalization to new

graphs

41
Tasks on nodes

 Labeling nodes in a graph

(clustering)
 Find topic of a research paper
(CORA, etc)
 Find bots in a social network
 … ?
 Labeling new nodes
 Perform regression

42
Tasks on edges
 Find relationships
 Contact map of aminoacids (Alphafold)
 Contact suggestion (social network)
 ETA for directions (regression)
 Relationships between segments in pictures
 ...

G. Zhu et al., “Scene Graph Generation: A Comprehensive Survey.” arXiv, Jun. 22, 2022. doi: 10.48550/arXiv.2201.00443.
43
Tasks on graphs
 Predict properties of graphs
 Chemical properties (solubility, carcinogenic, Alter
possible drug)
 Classification of the research field in an ego Ego
network
 ...

Creat ed by adindar
from t he Noun Project
Created by misirlou
from the Noun Project

44
Question break
Roadmap

Graphs are everywhere

➔ Complex data structures
➔ Basics of graph theory

Learning on Graphs
➔ Graph embedding
➔ Transductive and inductive learning
➔ Tasks on graph learning

A few examples
➔ Taxonomy of methods
➔ Graph convolution
➔ Message passing
➔ Graph Transformer
Taxonomy of methods

I. Chami, S. Abu-El-Haija, and B. Perozzi,

“Machine Learning on Graphs: A Model and Comprehensive Taxonomy”.
49
Graph convolution

 Just like for images we can learn from neighborhood with a convolution.
Step n Step n+1

 A bit more complex since the number of neighbors is unlikely to be

constant.
 We want the operator to be permutation invariant.

50
Graph convolution
 Several steps are needed to
retrieve information for distant
nodes.
 For large graphs → a cutoff
 It is possible to use a virtual
node connected to all other
nodes. But in practice this
becomes quickly intractable.

51
Message passing
 We have embeddings for each part of
the graph (possibly different vector
sizes).
 Each part can learn from the others via a
transformation.
Learnable
transformation Gn Gn+1
Edge embedding Node embedding

X = En En+1
Nn Nn+1

n0

 Information is aggregated to form a

message that the node/edge will send to
others.
'
n0 = (f
NN ,f EN ,f
GN
52
Alphafold transformer

J. Jumper et al., Highly accurate protein structure prediction with AlphaFold, Nature, vol. 596, no.
7873, Art. no. 7873, Aug. 2021, doi: 10.1038/s41586-021-03819-2.
53
Graph Transformer Network

Dwivedi, Bresson A Generalization of Transformer Networks to Graphs 2020, https://arxiv.org/abs/2012.09699

54
Question break
Resources

Libraries
 Pytorch Geometric  https://logconference.org/
 Deep Graph Library  https://ogb.stanford.edu/
 Graph Nets Tutorials
 Spektral
 ...  https://antoniolonga.github.io/Pytorch
_geometric_tutorials/
 https://docs.dgl.ai/tutorials/blitz

56
References
 Books
 Deep Learning on Graphs (Jiliang Tang and Yao Ma)
 Introduction to Graph Neural Networks (Introduction to Graph Neural Networks)
 Websites
 https://distill.pub/2021/gnn-intro/
 https://neptune.ai/blog/graph-neural-network-and-some-of-gnn-applications
 https://venturebeat.com/2021/10/13/what-are-graph-neural-networks-gnn/
 https://theaisummer.com/graph-convolutional-networks/
 https://towardsdatascience.com/node-embeddings-for-beginners-554ab1625d98
 Articles
● Chami, S. Abu-El-Haija, and B. Perozzi, “Machine Learning on Graphs: A Model and Comprehensive Taxonomy”.
● Zhou, Jie, et al. "Graph neural networks: A review of methods and applications." AI Open 1 (2020): 57-81.
● Scarselli, Franco, et al. "The graph neural network model." IEEE transactions on neural networks 20.1 (2008): 61-80.
● Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907 (2016).
● Perozzi, Bryan, Rami Al-Rfou, and Steven Skiena. "Deepwalk: Online learning of social representations." Proceedings of the 20th ACM SIGKDD
international conference on Knowledge discovery and data mining. 2014.
● Shlomi, Jonathan, Peter Battaglia, and Jean-Roch Vlimant. "Graph neural networks in particle physics." Machine Learning: Science and Technology 2.2
(2020): 021001.
● Duong, Chi Thang, et al. "On node features for graph neural networks." arXiv preprint arXiv:1911.08795 (2019).
● Dwivedi, Bresson "A Generalization of Transformer Networks to Graphs" 2020, https://arxiv.org/abs/2012.09699
● G. Zhu et al., “Scene Graph Generation: A Comprehensive Survey.” arXiv, Jun. 22, 2022. doi: 10.48550/arXiv.2201.00443

57
Message Passing Graph
Convolution Network
binary classification
Objective :
We want to classify if a molecule is active as an anti-
HIV drug.
Dataset :
OGB-molHIV
41000 molecules
https://ogb.stanford.edu/docs/graphprop/#ogbg-mol

58
Merci !

59
 Jeudi

Next, on Fidle :
07
Mars
à 14h00
L’IA
comme
un outil

Réalisation, technique et mise en scène :

Agathe, Baptiste et Yanis - UGA/DAPI
Thibaut, Kamel, Léo - IDRIS 60
 Jeudi
7
Mars
à 14h00

Next, on Fidle :
? ? ? ?
L’IA
comme
un outil

Réalisation, technique et mise en scène :

Agathe, Baptiste et Yanis - UGA/DAPI
Thibaut, Kamel, Léo - IDRIS 61
 Jeudi
7
Mars
à 14h00

Next, on Fidle :

L’IA
comme
un outil

Réalisation, technique et mise en scène :

Agathe, Baptiste et Yanis - UGA/DAPI
Thibaut, Kamel, Léo - IDRIS 62
 Jeudi

Next, on Fidle :
07
Mars
à 14h00
L’IA
comme
un outil

Réalisation, technique et mise en scène :

Agathe, Baptiste et Yanis - UGA/DAPI
Thibaut, Kamel, Léo - IDRIS 63