Graph Based Data

Science
Opportunities, challenges and techniques
Outline
• Graphs for Data Representation
• Overview of Graph Based Data Science
• Graph Analysis Algorithms
• Graph Aware ML/AI techniques
• Graph Kernels
• Graph Embeddings
• Graph Neural Networks

• Hands-on graph-based ML/AI example

Graphs for Data Representation1,2,46
A data graph is a mathematical representation of entities
(nodes) and their relationships (edges).

• Why data graphs?

• they represent data intuitively
• they make it easier to explore the data
• a lot of real-world examples are already graph structured**
Graphs for Data Representation1,2
Examples of real-world graph data:
• WWW, Internet [55]
• Social Networks
• Citation networks
• Protein interaction networks
• Communication networks
• Word co-occurrences networks
• Food webs
• Financial networks
• Economic networks
• Knowledge graphs
• ……
Overview of Graph Based Data
Science3
Graph based data science combines querying, statistics and graph
algorithms to power ML/AI applications by leveraging the relationships
and topology of graph data

• Graph Analytics: uses queries and graph statistics/algorithms** to

answer such questions as:
• how many customers are in a marketing graph?
• are there any fraud rings in the transaction data?
• who’s the most important influencer in a network?

• Graph Enhanced ML and AI: uses queries, graph statistics/algorithms

and specialized graph aware ML/AI techniques to describe the global
topology or connectivity of a data set. The results define graph-based
features that are used as input to downstream ML/AI tasks
Overview of Graph Based Data
Science3
Graph Queries: retrieve an explicit pattern within the dataset
e.g.
• fetch the names of George’s co-authors
• compute the average salary of a company’s employees

Graph Statistics: are based on local properties of the graph

nodes:
• Graph size
• Graph density
• Degree distribution
• Local clustering coefficient
Overview of Graph Based Data
Science3
Graph Algorithms: employ graph traversals

• Connected Components: identify strongly connected subgraphs

• Community Detection: partition a graph based on relationships
• Centrality: reveal which nodes are important based on graph topology
• Node Similarity: identify similar nodes based on their neighbors or
properties
• Pathfinding: find the shortest or most efficient paths to traverse a graph
• Link Prediction: predict unobserved or future relationships between
nodes**

Some of the graph algorithms employ ML techniques themselves!

Graph Algorithms
Connected Components5
• Undirected Graph:
• Breadth First Search
• Depth First Search
• Directed Graph:
• Weakly Connected Components: discard edge directions and proceed as before
• Strongly Connected Components: Kosaraju's algorithm57

• Use Cases
• Image Processing29
• Entity Resolution/deduplication
• Cybersecurity30
• Fraud Detection
• etc.
Graph Algorithms
Community Detection11

• Algorithms:
• Louvain6
• Label Propagation7
• many more8

• Use Cases:
• Social networks analysis: e.g., extracting topics from online social platforms 9
• Criminology: identification of groups in criminal networks
• Public Health: discover dynamics of certain groups susceptible to an epidemic
disease
• Customer Segmentation: identify groups of people that are using the same
financial service
• Bioinformatics: investigating the human brain and finding hierarchical
community structures within the brain’s functional network 10
•
Graph Algorithms
Node Similarity

Based on:
• Node’s own features
• Node’s own and neighbors’ ID/features
• Similar neighbors (recursive definition)
• Graph topology
• Any combination of the above

Use cases:
• Entity matching/resolution31
• Recommendations (e.g., ‘Customers who brought this item also
brought..’)
• etc.
Graph Algorithms
Centrality11

• Variations:
• Degree Centrality: nodes with lots of direct connections
• Closeness Centrality: nodes that can reach other nodes with few hops
• Betweenness Centrality: nodes that sit on the shortest path of lots of
pairs of other nodes
• Eigenvector Centrality: nodes that are transitively connected to other
important nodes

• Use Cases:
• Identification of social media influencers12,13
• Fighting fraud and terrorism14,15,16
• Predicting flow in traffic, delivery and telecommunications systems 17,18
• etc.
Graph Algorithms
Path Finding5
• Algorithms
• Shortest Path
• A*
• Yen’s k-shortest paths
• All pairs shortest path
• Single source shortest path
• Minimum Spanning Tree
• Random Walks

• Use cases
• Directions between two physical locations
• Degrees of separation between people
• Travel Planning20, 21
• Financial Analysis19
• Graph Embeddings**
• etc.
Graph Aware ML/AI
Techniques
Graph Kernels32
Graph Kernels can be intuitively understood as functions
quantifying the similarity of pairs of graphs.

Kernel based on shortest

paths:
G1 G2 G3 Sim (G1, G2) = 0.98
Sim (G2, G3) = 0.90
Sim (G1, G3) = 0.86
Graph Kernels32,33

What is a graph kernel?

• Map two graphs g1 and g2 using mapping φ into feature space Η

• Measure their similarity in Η as <φ(g1), φ(g1)>
• Actualize kernel trick: devise kernel function k such k(g1, g2)
= <φ(g1), φ(g1)>
Graph Kernels32,33
Requirement for graph kernel functions:

• Positive definite (in order to be a valid kernel function)

• Expressive
• define a non-trivial measure of similarity on graphs
• represent subgraph features that allow to tell if the topology, node and
edge labels of two graphs are similar
• Applicable to a wide range of graphs
• Efficient to compute
Graph Kernels32,33
Graph Kernel variants34: graph kernels take the form of an
aggregation over metrics that describe different aspects of
the graph topology:

• based on local structure: two graphs are similar if they are

composed of nodes with similar neighborhoods
• graph components: compare graphs by identifying the best
possible matching of the components making up the graphs
(nodes, labels, edges, subgraphs etc.)
• paths and walks: compare sequences of node or edge
attributes encountered through graph traversals (e.g.,
shortest paths, random walks)
Graph Kernels32,33
Applications of Graph Kernels:

• ML/AI
• Allow kernelized ML algorithms (e.g., SVMs) to work directly on graphs
• Can be employed to learn node embeddings41**

• Chemoinformatics : predicting the toxicity of chemical molecules37

• Drug discovery: virtual screening36
• Bioinformatics : protein function prediction35
• Web data mining and Social Networks:
• node classification39, 40
• link prediction38, 40
• etc.
Graph Embeddings22
Embedding in ML/AI consists of representing complex
objects like text, images or graphs with continuous vectors
of a reduced dimensionality compared to the
dimensionality of the dataset, while keeping the most
important information about them.
Graph Embeddings22
Graph embeddings should capture the graph topology,
node-to-node relationships, and content information.

• Node embeddings
• Whole graph embeddings [23]
Node Embeddings24
Each node is represented with a continuous vector of a
fixed dimensionality. They are used to perform
visualization or prediction at the node level, e.g., link
prediction.

Popular techniques:
• Random Walk Based Approaches:
• DeepWalk42 : uses random walks to produce embeddings
• node2vec 43 : modification of DeepWalk that provides control over the
explored neighborhoods
• Deep Learning Approaches
• GraphSAGE 45: an inductive scalable technique
• Graph Neural Networks**
Node Embeddings24
DeepWalk42

1. The graph is sampled using random walks

2. A skip-gram (MLP) network accepts a node from the random walk
as a one-hot vector as an input and maximizes the probability for
predicting neighbor nodes
3. The node embedding is the output of the last hidden layer of the
network
Node Embeddings24
node2vec43: modified random walks to better user
exploration requirements

Each random walk accepts two (additional) parameters:

• Q: controls the likehood to discover the undiscovered part of the
graph
• P: controls the likehood to return to the previous node
Node Embeddings24, 47, 48
GraphSAGE45: learns aggregator functions that can induce
the embedding of a previously unseen node (thus
inductive technique) given its features and neighborhood.
It consists of two main components:

• Context construction
• Information Aggregation
Node Embeddings24, 47, 48
GraphSAGE45
• Context construction
• the neighborhood of a node consists of the nodes up to K hopes away form
that node

• Assumption: nodes that reside in the same neighborhood should have similar
embeddings
Node Embeddings24, 47, 48, 49
GraphSAGE45
• Information Aggregation
• instead of learning node embeddings GraphSAGE learns
aggregation functions
• aggregation functions accept a neighborhood as input and
combine each neighbor’s embedding with weights to create a
neighborhood embedding:
• LSTM on a random permutation of the nodes in a neighborhood
• average
• MLP followed by max pooling
Whole Graph Embeddings24
The entire graph is represented with a single vector. They
are used to make predictions at the graph level or to
compare or visualize graphs, e.g., in the comparison of
chemical structures.

• graph2vec25: unsupervised technique with relatively stable

performance

• Many other techniques but no clear winners:

• sub2vec (embed subgraphs)27: produces subgraph embeddings
• UGRAPHEMB28: an inductive method
Whole Graph Embeddings24
graph2vec25: views the graph as a document and the
rooted subgraphs around every node as words that
compose the document. It is trained to maximize the
probability of predicting subgraphs:
Graph Neural Networks50,51,53,55
Graph Neural Networks (GNNs) are connectionist models
that capture the relationships represented in the graphs:

• there is one GNN ‘recurrent unit’ for each node in the

graph: each unit maintains stateful information
(embeddings) that captures
• the properties of the related node
• the neighborhood properties of that node
Graph Neural Networks50,51,53,55
Graph Neural Networks (GNNs) are connectionist models that
capture the relationships represented in the graphs:

• an MLP for each edge learns the importance/properties of

the underlying relation
Graph Neural Networks50,51,53,55
Graph Neural Networks (GNNs) are connectionist models
that capture the relationships represented in the graphs:

• neighborhood aggregation is employed to identify the

dependencies between the nodes of graphs
Graph Neural Networks50,51,53,55
Graph Neural Networks (GNNs) are connectionist models
that capture the relationships represented in the graphs:

• once neighborhood aggregation is performed the node

embeddings capture more accurately the information
about their own features and that of their neighboring
nodes
• the vector H that results by taking the sum of all the
node embeddings represents the whole graph
Graph Neural Networks52,53,54
Applications of GNNs:

• ML/AI
• Node classification
• Link Prediction

• Physical Systems: model Interaction Networks and predict future trajectories

• Chemistry:
• more flexible means to predict the properties of a new molecule by learning from example
molecules
• predict side effects due to drug interactions
• Image Classification: alternative to CNNs to achieve zero-shot or few-shot learning
• Combinatorial Optimization: have shown promise as alternatives to relaxation-
based techniques
• Social graphs: content recommendation
• Transportation: traffic prediction
• etc.
 Hands-on graph-based ML/AI
example
https://bitbucket.org/diip20201/tutorials/src/master/graph_based_data_science/
References
1. https://towardsdatascience.com/graph-databases-whats-the-big-deal-ec310b1bc0ed
2. https://snap.stanford.edu/data/
3. https://neo4j.com/blog/announcing-neo4j-for-graph-data-science/#:~:text=Graph%20data%20science%20uses%20multidisciplinary,artificial%20intelligence%20(AI)%20application
s
.
4. https://neo4j.com/blog/how-graphs-enhance-artificial-intelligence/
5. Alfred V. Aho, John E. Hopcroft and Jeffrey D. Ullman. Data Structures and Algorithms. Addison-Wesley, 1983
6. H. Lu, M. Halappanavar and A. Kalyanaraman: Parallel Heuristics for Scalable Community Detection, Parallel Computing, vol. 47, pp. 19-37, 2015
7. X. Zhu and Z. Ghahramani: Learning from Labeled and Unlabeled Data with Label Propagation, CMU CALD tech report CMU-CALD-02-107, 2002
8. J. Leskovec, K. J. Lang and M. W. Mahoney: Empirical Comparison of Algorithms for Network Community Detection, WWW 2010
9. G. S. Kido, R. A. Igawa and R. C. Garcia: Topic Modeling based on Louvain method in Online Social Networks, SBSI 2016
10. D. Meunier, R. Lambiotte, A. Fornito, K. D. Ersche, and Edward T. Bullmore: Hierarchical Modularity in Human Brain Functional Networks, Frontiers in Bioinformatics, 2009
11. https://neo4j.com/developer/graph-data-science/centrality-graph-algorithms/
12. https://www.brandwatch.com/blog/react-influential-men-and-women-2017/
13. S. Wu, L. Gong, W. Rand and L. Raschid, Making recommendations in a microblog to improve the impact of a focal user, SSRN Electronic Journal, 2002
14. V. E. Krebs: Mapping Networks of Terrorist Cells, Connections 2002
15. P. Bangcharoensap, H. Kobayashi, N. Shimizu, S. Yamauchi and T. Murata: Two Step graph-based semi-supervised Learning for Online Auction Fraud Detection, ECML PKDD 2015
16. C. Morselli and J. Roy: Brokerage qualifications in ringing operations, Criminology 2008
17. S. P. Borgatti: Centrality and Network Flow, Social Networks 2005
18. B. Jiang, S. Zhao and J. Yin: Self-organized Natural Roads for Predicting Traffic Flow: A Sensitivity Study, Journal of Statistical Mechanics Theory and Experiment 2007
19. T. T. Chitenderu: The Random Walk Theory And Stock Prices: Evidence From Johannesburg Stock Exchange, IBER 2014
References
20. T. Chondrogiannis, P. Bouros, J. Gamper and U. Leser: Alternative Routing: k-Shortest Paths with Limited Overlap, SIGSPATIAL 2015
21. L. Fitina, J. Impal, V. Uiari, N. Murki and E. Goodyear: An Application of Minimum Spanning Trees to Travel Planning, DWU Research Journal 2010
22. https://medium.com/@st3llasia/graph-embedding-techniques-7d5386c88c5
23. W. L. Hamilton, R. Ying and J. Leskovec: Representation Learning on Graphs: Methods and Applications, IEEE Data Engineering Bulletin 2007
24. https://towardsdatascience.com/graph-embeddings-the-summary-cc6075aba007
25. A. Narayanan, M. Chandramohan, R. Venkatesan, L. Chen, Y. Liu, and S. Jaiswal: graph2vec: Learning Distributed Representations of Graphs, MLG 2017
26. M. Niepert, M. Ahmed, K. Kutzkov: Learning Convolutional Neural Networks for Graphs ICML 2016
27. B. Adhikari, Y. Zhang, N. Ramakrishnan and B. A. Prakash: Sub2Vec: Feature Learning for Subgraphs, PAKDD 2018
28. Y. Bai, H. Ding, Y. Qiao, A. Marinovic, K. Gu, T. Chen, Y. Sun and W. Wang: Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity, IJCAI 2019
29. W. Song, D. Wu, Y. Xi, Y. W. Park, and K. Cho, Motion-based skin region of interest detection with a real-time connected component labeling algorithm, Multimedia Tools and
Applications, 2016.
30. M. Yip, N. Shadbolt, and C. Webber, Structural analysis of online criminal social networks, Intelligence and Security Informatics, 2012
31. M. Pershina, M. Yakout, K. Chakrabati: Holistic Entity Matching Across Knowledge Graphs, BigData 2015
32. https://sites.cs.ucsb.edu/~xyan/tutorial/KDD08_graph_partII.pdf
33. https://d-nb.info/985213434/34
34. https://arxiv.org/pdf/1903.11835.pdf
35. K. M. Borgward, C. S. Ong, S. Schönauer, S. V. N. Vishwanathan, A.J Smola and H.-P. Krigel: Protein function prediction via graph kernels, Bioinformatics 2005
36. M. P. Preeja and S. Kp: Walk-based Graph Kernel for Drug Discovery: A Review, International Journal of Computer Applications 2014
37. http://www.normastic.fr/wp-content/uploads/2019/05/presentation-03.05.2019.pdf
References
38. W. Yuan, K. He, D. Guan, L. Zhou and C. Li: Graph kernel based link prediction for signed social networks, Information Fusion 2019
39. N. Bui and V. Honavar: Labeling Actors in Social Networks Using a Heterogeneous Graph Kernel, SBP 2014
40. U. Losch, S. Bloehorn and A. Rettinger: Graph Kernels for RDF Data, ESWC 2012
41. Y. Tian, L. Zhao, X. Peng and D. N. Metaxas: Rethinking Kernel Methods for Node Representation Learning on Graphs, NeurIPS 2019
42. B. Perozzi, R. Al-Rfou and S. Skiena: Deep Walk: Online Learning of Social Representations, KDD 2014
43. A. Grover and J. Leskovec: node2vec: Scalable Feature Learning for Networks, KDD 2016
44. D. Wang, P. Cui, W. Zhu: Structural Deep Network Embedding, KDD 2016
45. W. L. Hamilton, R. Ying, and J. Leskovec: Inductive Representation Learning on Large Graphs, NIPS 2017
46. https://www.kaggle.com/ferdzso/knowledge-graph-analysis-with-node2vec
47. https://towardsdatascience.com/an-intuitive-explanation-of-graphsage-6df9437ee64f
48. http://snap.stanford.edu/graphsage/
49. https://medium.com/analytics-vidhya/ohmygraphs-graphsage-and-inductive-representation-learning-ea26d2835331
50. https://towardsdatascience.com/a-gentle-introduction-to-graph-neural-network-basics-deepwalk-and-graphsage-db5d540d50b3
51. F. Scarselli, M. Gori, A. Tsoi, M. Hagenbuchner and G. Monfardini: The graph neural network model, IEEE Transactions on Neural Networks, 20(1) 2009
52. https://towardsdatascience.com/https-medium-com-aishwaryajadhav-applications-of-graph-neural-networks-1420576be574
53. J. Zhou, G. Cui, Z. Zhang, C Yang, Z. Liu, L. Wang, C. Li, M. Sun: Graph Neural Networks: A Review of Methods and Applications, ArXiv 2018
54. https://blog.fastforwardlabs.com/2019/10/30/exciting-applications-of-graph-neural-networks.html
55. https://medium.com/dair-ai/an-illustrated-guide-to-graph-neural-networks-d5564a551783
56. N. Shervashidze, P. Schweitzer, E. J. van Leeuwen, K. Mehlhorn and K. M. Borgwardt: Weisfeiler-Lehman Graph Kernels, Journal of Machine Learning Research 12 (2011)
57. https://en.wikipedia.org/wiki/Kosaraju%27s_algorithm