towards_analytics_query_engine

Analytics Query Engine
Nantia Makrynioti and Vasilis Vassalos
Athens University of Economics and Business

MEDAL 2016, Bordeaux, France
TOWARDS AN

• Huge amount of available data

• Decrease of storage cost

• Great use of systems facilitating data analytics in a
distributed fashion
Towards an Analytics Query EngineNantia Makrynioti & Vasilis Vassalos
INTRODUCTION

CURRENT STATE OF THE ART
• Libraries of algorithms

• Systems that provide operators for developing
distributed algorithms

✦ Combination of procedural and declarative
programming

✦ Integration of declarative operators to imperative
languages

✦ Closer to the style of statistical computing languages

Implementations of machine learning algorithms on top
of a given distributed framework

final
LogisticRegressionModel
model
=
new

LogisticRegressionWithLBFGS()

.setNumClasses(10)

.run(training.rdd());

Examples: Apache Mahout, MLlib, MADlib
LIBRARIES OF ALGORITHMS
+ Easy to use
- Too coarse grained

A programming model between SQL and Map Reduce

good_urls
=
FILTER
urls
BY
page
rank
>
0.2;

groups
=
GROUP
good_urls
BY
category;

Examples: Jaql, Pig Latin, Stratosphere’s Sopremo layer, uSQL
SYSTEMS OF OPERATORS (1)
A sequence of steps, with
each step performing a single high-
level transformation
Use of variables to store
intermediate results
+ Closer to data analysis users, yet
fairly declarative
- Limited support for iteration

Extensions of the MapReduce model

val
textFile
=
sc.textFile("hdfs://...")

val
df
=
textFile.toDF("line")

val
errors
=
df.filter(col("line").like("%ERROR%"))

errors.count()

Examples: DryadLINQ, Stratosphere’s PACT, Spark,
Tupleware
Richer set of operators
Arbitrary
data ﬂow graphs
+ Iteration support
- Considerable amount of user
deﬁned code

R-like programming models

while(i
<
20)
{

H
=
H
*
(t(W)
%*%
V)/(
t(W)
%*%
W
%*%
H);

W
=
W
*
(V
%*%
t(H)/(W
%*%
H
%*%
t(H));

i
=
i
+
1;

}

Examples: MLI API, SystemML
Support for matrix
operations
+ Easier to express ML algorithms due to linear
algebra operations
- Still closer to imperative programming

good_urls
=
FILTER
urls
BY
page
rank
>
0.2;

groups
=
GROUP
good_urls
BY
category;

val
points
=
spark.textFile(…).map(parsePoint).cache()

var
w
=
Vector.random(D);

for(i
<-‐
1
to
ITERATIONS)
{

val
grad
=
spark.accumulator(new
Vector(D));

for(p<-‐points)
{

val
s
=
(1/(1+exp(-‐p.y*(w
dot
p.x)))-‐1)*p.y

grad
+=
s
*
p.x

}

w
-‐=
grad.value

WHAT’S MISSING?
Machine Learning
Algorithms: very close
to imperative
languages
Relational
queries: very close to
SQL style

Can we increase declarativity in 
speciﬁcation of ML algorithms?
WHAT’S MISSING?

VISION
An analytics query engine that would enable
declarative speciﬁcation, operator based execution
and cost-based optimization of machine learning
algorithms on distributed processing platforms.

• A language with a SQL-like syntax

• Use of Datalog to express ML algorithms in a
declarative manner

✦ Translation of existing programming models for analytics
into Datalog programs [13]

✦ Extension of Datalog to cover a wider range of machine
learning tasks [14]
DECLARATIVE SYNTAX

Hypothesis

Cost function

Optimization function

Repeat { }
} Towards an Analytics Query EngineNantia Makrynioti & Vasilis Vassalos
DECOMPOSITION OF
LINEAR REGRESSION
✓j := ✓j a
@
@✓j
J(✓)
h✓(x) = ✓T
x
J(✓) =
1
2m
mX
1
(h✓(x(i)
) y(i)
)2

DEFINE ML OPERATORS
Based on the example of Linear Regression we need:

✦ Linear algebra operators

✦ Aggregation operators

✦ Iteration

✦ Optimization functions are also good candidates to
deﬁne operators

• Query rewriting as in databases

✦ Operators for ML algorithms would result in clearer
semantics

• Compiler optimizations: function inlining, SIMD
vectorization
OPTIMIZATION TECHNIQUES
Mostly
applicable to relational
operators
Useful for user
deﬁned code
More opportunities for
query rewriting

• Optimization techniques from machine learning

✦ Speed up training with large datasets using
stochastic gradient descent instead of batch
gradient descent

✦ Feature scaling for faster convergence

Note that these optimizations may produce slightly
diﬀerent output
OPTIMIZING PROGRAMS

• Deﬁne a language with a declarative syntax

• Translate this language into a set of operators, which
express a wide range of machine learning tasks

• Based on their semantics, deﬁne algebraic properties
for query optimization

• Implement these operators on a distributed
processing framework
ACTION PLAN

Thank you!
Questions?

1. Apache mahout. http://mahout.apache.org/.

2. X. Meng et al. MLlib: Machine learning in apache spark. CoRR, abs/1505.06807, 2015.

3. J. Cohen et al. Mad skills: New analysis practices for big data. Proc. VLDB Endow., 2(2):1481–1492, Aug. 2009.

4. K. S. Beyer et al. Jaql: A scripting language for large scale semistructured data analysis. PVLDB, 4(12)
1272-1283, 2011

5. C.Olston et al. Pig latin: A not-so-foreign language for data processing. In Proceedings of the 2008 ACM
SIGMOD International Conference on Management of Data, SIGMOD ’08, pages 1099–1110, 2008.

6. A. Alexandrov et al. The stratosphere platform for big data analytics. The VLDB Journal, 23(6):939-964, Dec.
2014.

7. U-SQL. http://usql.io/.

8. A. Crotty et al. Tupleware: Redeﬁning modern analytics. CoRR, abs/1406.6667, 2014.

9. A. Ghoting et al. SystemML: Declarative machine learning on MapReduce. In Proceedings of the 2011 IEEE
27th International Conference on Data Engineering, ICDE ’11, pages 231–242, 2011.

10. E. R. Sparks et al. MLI: An API for distributed machine learning.

11. Y. Yu et al. DryadLINQ: A system for general-purpose distributed data-parallel computing using a high-level
language. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation,
OSDI’08, pages 1–14, 2008.

12. M. Zaharia et al. Spark: Cluster computing with working sets. In Proceedings of the 2Nd USENIX Conference
on Hot Topics in Cloud Computing, HotCloud’10, pages 10–10, 2010.

13. Y. Bu et al. Scaling Datalog for Machine Learning on Big Data. CoRR, abs/1203.0160, 2012.

14. V. Barany et al., Declarative Statistical Modeling with Datalog. CoRR abs/1412.2221, 2014. 
REFERENCES

towards_analytics_query_engine

Recommended

More Related Content

What's hot (19)

Similar to towards_analytics_query_engine (20)

towards_analytics_query_engine