SlideShare a Scribd company logo
Quintor 
Web-scale op basis van 
Hadoop 
JOHAN 
TILLEMA 
CEO
Quintor 
Quintor 
JAVA 
.NET 
MOBILE 
GRONINGEN 
AMERSFOORT 
78 MEDEWERKERS 
AGILE/SCRUM 
ATLASSIAN 
MS TFS 
ANALYSE 
2005 
SOFTWARE 
DEVELOPERS
Quintor 
Big data 
= 
Web-Scale
Facebook 
> 300PB aan data 
1 miljoen foto's per seconde 
100 miljard page hits per dag 
Slaat 1000 miljard items op 
(foto's, events, statusupdates…)
Twitter 
500 miljoen tweets per dag 
300.000 queries per seconde 
Tientallen petabytes structured data 
Tweet binnen vijf seconden bij volgers
Whatsapp 
65 miljard berichten per dag 
450 miljoen actieve gebruikers 
150 miljoen tegelijk online 
150Gb/s traffic 
Nagenoeg real-time
1/3e van het internet verkeer in Amerika 
Streamt 1 miljard uur content per maand 
30 miljoen 'plays' per dag 
Enkele petabytes aan videomateriaal en metadata
Quintor 
Cheap hardware 
clusters 
Lineair schaling 
Parallel computing
Quintor 
Quintor 
Summercamp
Quintor 
RENE BOERE 
CTO
Quintor 
Web-scale technieken
Quintor 
Big Willem
Quintor 
Reactive streams
Quintor 
Hadoop @ hardware clusters 
YARN 
HDFS 
(Hadoop Distributed File system) 
Ambari 
Zookeeper 
Map 
Reduce Spark Storm HBase Pig … 
Ganglia / Nagios
Quintor 
CHRIS 
QUACH 
JAVA 
SPECIALIST
Quintor 
Pinguin demo
Quintor 
Hadoop @ hardware clusters 
YARN 
HDFS 
(Hadoop Distributed File system) 
Ambari 
Zookeeper 
Map 
Reduce Spark Storm HBase Pig … 
Ganglia / Nagios
Quintor 
Breadth First Search 
7.5 
7.4 7.6 
6.4 7.3 8.4 8.6 
7.7 
8.3 9.6 
8.7 
9.3 9.4 9.5 9.7
Quintor 
7.5 0 7.4,7.6 
7.4 MAX_VALUE 6.4,7.3,7.5,8.4 
7.6 MAX_VALUE 7.5,7.7,8.6 
6.4 MAX_VALUE 7.4 
7.3 MAX_VALUE 7.4,8.3 
8.4 MAX_VALUE 7.4,8.3 
8.3 MAX_VALUE 7.3,9.3 
9.3 MAX_VALUE 8.3,9.4 
9.4 MAX_VALUE 8.4,9.3,9.5
Quintor 
Storm topology 
InputSpout 
PersistBolt 
ProcessBolt
Quintor 
Breadth First Search 
7.5 
7.4 7.6 
6.4 7.3 8.4 8.6 
7.7 
8.3 9.6 
8.7 
9.3 9.4 9.5 9.7
Quintor 
Breadth First Search 
7.5 
7.4 7.6 
6.4 7.3 8.4 8.6 
7.7 
8.3 9.6 
8.7 
9.3 9.4 9.5 9.7
Quintor 
Storm topology 
InputSpout 
PersistBolt 
IntermediateBolt 
ProcessBolt
Quintor 
BIG DATA VEREIST WEB-SCALE DENKEN 
WEB-SCALE VERANDERT ALLES 
DE TECHNOLOGIE WORDT NU ONTWIKKELD
Quintor 
10:50 
VERVOLGSESSIE 
SALVADOR 
27 NOVEMBER 
HANDS-ON WEBSCALE 
STAND RECHTS NAAST DE 
INGANG 
14:30 
JSF -> SPA MIGRATIE 
REMBRANDT
“Once you stop learning, you start dying” 
Quintor 
Bedankt voor jullie aandacht 
- Albert Einstein 
Quintor
Quintor 
Bedankt voor jullie aandacht

More Related Content

Viewers also liked (10)

Reactieve applicaties allard buijze
Reactieve applicaties   allard buijzeReactieve applicaties   allard buijze
Reactieve applicaties allard buijze
NLJUG
 
Decoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan SchrijverDecoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
NLJUG
 
Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...
NLJUG
 
Scratching the Surface with JavaFX
Scratching the Surface with JavaFXScratching the Surface with JavaFX
Scratching the Surface with JavaFX
NLJUG
 
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter DanesUsing Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
NLJUG
 
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
NLJUG
 
Real-World Scala Design Patterns
Real-World Scala Design PatternsReal-World Scala Design Patterns
Real-World Scala Design Patterns
NLJUG
 
Shootout! template engines on the jvm
Shootout! template engines on the jvmShootout! template engines on the jvm
Shootout! template engines on the jvm
NLJUG
 
Opening J-Fall 2013
Opening J-Fall 2013Opening J-Fall 2013
Opening J-Fall 2013
NLJUG
 
Akka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsAkka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based Applications
NLJUG
 
Reactieve applicaties allard buijze
Reactieve applicaties   allard buijzeReactieve applicaties   allard buijze
Reactieve applicaties allard buijze
NLJUG
 
Decoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan SchrijverDecoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
Decoding the airspace above you with Java and $7 hardware - Bert Jan Schrijver
NLJUG
 
Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...Speedy perception trumps speedy reception–smart asynchronous interactions - L...
Speedy perception trumps speedy reception–smart asynchronous interactions - L...
NLJUG
 
Scratching the Surface with JavaFX
Scratching the Surface with JavaFXScratching the Surface with JavaFX
Scratching the Surface with JavaFX
NLJUG
 
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter DanesUsing Docker to Develop, Test and Run Maven Projects - Wouter Danes
Using Docker to Develop, Test and Run Maven Projects - Wouter Danes
NLJUG
 
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
Gebruik dezelfde Docker container voor Java applicaties tijdens ontwikkelen e...
NLJUG
 
Real-World Scala Design Patterns
Real-World Scala Design PatternsReal-World Scala Design Patterns
Real-World Scala Design Patterns
NLJUG
 
Shootout! template engines on the jvm
Shootout! template engines on the jvmShootout! template engines on the jvm
Shootout! template engines on the jvm
NLJUG
 
Opening J-Fall 2013
Opening J-Fall 2013Opening J-Fall 2013
Opening J-Fall 2013
NLJUG
 
Akka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based ApplicationsAkka in Practice: Designing Actor-based Applications
Akka in Practice: Designing Actor-based Applications
NLJUG
 

More from NLJUG (20)

The future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris QuachThe future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
NLJUG
 
Real-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard BuijzeReal-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard Buijze
NLJUG
 
The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...
NLJUG
 
Performance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen BorgersPerformance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen Borgers
NLJUG
 
Introduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus JuraIntroduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus Jura
NLJUG
 
Workshop angular dart presentatie - Atos
Workshop angular dart presentatie - AtosWorkshop angular dart presentatie - Atos
Workshop angular dart presentatie - Atos
NLJUG
 
Rethink your architecture - Marten Deinum
Rethink your architecture - Marten DeinumRethink your architecture - Marten Deinum
Rethink your architecture - Marten Deinum
NLJUG
 
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopperEvolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
NLJUG
 
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
NLJUG
 
Apache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn DashorstApache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn Dashorst
NLJUG
 
Opening - Bert Ertman
Opening - Bert ErtmanOpening - Bert Ertman
Opening - Bert Ertman
NLJUG
 
Returning the right results - Jettro Coenradie
Returning the right results - Jettro CoenradieReturning the right results - Jettro Coenradie
Returning the right results - Jettro Coenradie
NLJUG
 
Reactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn BlankestijnReactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
NLJUG
 
Event-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander MakEvent-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander Mak
NLJUG
 
Setting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas GeerdinkSetting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas Geerdink
NLJUG
 
A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld
NLJUG
 
ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...
NLJUG
 
Data Science with R for Java Developers
Data Science with R for Java DevelopersData Science with R for Java Developers
Data Science with R for Java Developers
NLJUG
 
Modular JavaScript
Modular JavaScriptModular JavaScript
Modular JavaScript
NLJUG
 
JVM bytecode engineering 101
JVM bytecode engineering 101JVM bytecode engineering 101
JVM bytecode engineering 101
NLJUG
 
The future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris QuachThe future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
The future of Web-Scale - Johan Tillema, Rene Boere & Chris Quach
NLJUG
 
Real-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard BuijzeReal-time user interfaces - sosm gewoon makkelijker - Allard Buijze
Real-time user interfaces - sosm gewoon makkelijker - Allard Buijze
NLJUG
 
The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...The end of traditional enterprise IT - ING's journey to the next generation I...
The end of traditional enterprise IT - ING's journey to the next generation I...
NLJUG
 
Performance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen BorgersPerformance van Java 8 en verder - Jeroen Borgers
Performance van Java 8 en verder - Jeroen Borgers
NLJUG
 
Introduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus JuraIntroduction to Reactive with Play and Akka - Markus Jura
Introduction to Reactive with Play and Akka - Markus Jura
NLJUG
 
Workshop angular dart presentatie - Atos
Workshop angular dart presentatie - AtosWorkshop angular dart presentatie - Atos
Workshop angular dart presentatie - Atos
NLJUG
 
Rethink your architecture - Marten Deinum
Rethink your architecture - Marten DeinumRethink your architecture - Marten Deinum
Rethink your architecture - Marten Deinum
NLJUG
 
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopperEvolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
Evolutionary Algorithms: the key to solving complex Java puzzles! - Bas knopper
NLJUG
 
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
Modularity and Domain Driven Design; a killer Combination? - Tom de Wolf & St...
NLJUG
 
Apache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn DashorstApache Wicket: 10 jaar en verder - Martijn Dashorst
Apache Wicket: 10 jaar en verder - Martijn Dashorst
NLJUG
 
Opening - Bert Ertman
Opening - Bert ErtmanOpening - Bert Ertman
Opening - Bert Ertman
NLJUG
 
Returning the right results - Jettro Coenradie
Returning the right results - Jettro CoenradieReturning the right results - Jettro Coenradie
Returning the right results - Jettro Coenradie
NLJUG
 
Reactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn BlankestijnReactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
Reactive programming met Java 8 en Java EE 7 - Martijn Blankestijn
NLJUG
 
Event-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander MakEvent-sourced architectures with Akka - Sander Mak
Event-sourced architectures with Akka - Sander Mak
NLJUG
 
Setting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas GeerdinkSetting up a mini big data architecture, just for you! - Bas Geerdink
Setting up a mini big data architecture, just for you! - Bas Geerdink
NLJUG
 
A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld A Storm of Drones - Dennis Vredeveld
A Storm of Drones - Dennis Vredeveld
NLJUG
 
ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...ING : How top quality software and state-of-the-art technology leads to conti...
ING : How top quality software and state-of-the-art technology leads to conti...
NLJUG
 
Data Science with R for Java Developers
Data Science with R for Java DevelopersData Science with R for Java Developers
Data Science with R for Java Developers
NLJUG
 
Modular JavaScript
Modular JavaScriptModular JavaScript
Modular JavaScript
NLJUG
 
JVM bytecode engineering 101
JVM bytecode engineering 101JVM bytecode engineering 101
JVM bytecode engineering 101
NLJUG
 

Web-scale op basis van Hadoop en Akka Reactive Streams - Johan Tillema, Rene Boere & Chris Quach

Editor's Notes

  • #19: Nodes toevoegen aan queue Queue verwerken totdat deze leeg is, geen dubbele nodes Cluster, wat met de queue?
  • #22: Hadoop ontworpen om op verschillende hardware configuraties te draaien Afstand naar 9.4 is 4 en niet 5