Incanter Data Sorcery
6 likes2,170 views
The document discusses various functions for working with datasets in the Incanter library for Clojure. It describes how to create, read, save, select rows and columns from, and sort datasets. Functions are presented for building datasets from sequences, reading datasets from files and URLs, saving datasets to files and databases, selecting single or multiple columns, and filtering rows based on conditions. The document also provides an overview of the Incanter library and its various namespaces for statistics, charts, and other functionality.
![(use 'incanter.core)
(dataset ["x1" "x2" "x3"]
[[1 2 3]
[4 5 6]
[7 8 9]])
(to-dataset [{"x1" 1, "x2" 2, "x3" 3}
{"x1" 4, "x2" 5, "x3" 6}
{"x1" 7, "x2" 8, "x3" 9}])
(to-dataset [[1 2 3]
[4 5 6]
[7 8 9]])
(conj-cols [1 4 7] [2 5 8] [3 6 9])
(conj-rows [1 2 3] [4 5 6] [7 8 9])
Creating datasets create a small dataset](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-14-320.jpg)
![(use 'incanter.core)
(dataset ["x1" "x2" "x3"]
[[1 2 3]
[4 5 6]
[7 8 9]])
(to-dataset [{"x1" 1, "x2" 2, "x3" 3}
{"x1" 4, "x2" 5, "x3" 6}
{"x1" 7, "x2" 8, "x3" 9}])
(to-dataset [[1 2 3]
[4 5 6]
[7 8 9]])
(conj-cols [1 4 7] [2 5 8] [3 6 9])
(conj-rows [1 2 3] [4 5 6] [7 8 9])
convert a sequence of mapsCreating datasets](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-15-320.jpg)
![(use 'incanter.core)
(dataset ["x1" "x2" "x3"]
[[1 2 3]
[4 5 6]
[7 8 9]])
(to-dataset [{"x1" 1, "x2" 2, "x3" 3}
{"x1" 4, "x2" 5, "x3" 6}
{"x1" 7, "x2" 8, "x3" 9}])
(to-dataset [[1 2 3]
[4 5 6]
[7 8 9]])
(conj-cols [1 4 7] [2 5 8] [3 6 9])
(conj-rows [1 2 3] [4 5 6] [7 8 9])
convert a sequence of sequencesCreating datasets](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-16-320.jpg)
![(use 'incanter.core)
(dataset ["x1" "x2" "x3"]
[[1 2 3]
[4 5 6]
[7 8 9]])
(to-dataset [{"x1" 1, "x2" 2, "x3" 3}
{"x1" 4, "x2" 5, "x3" 6}
{"x1" 7, "x2" 8, "x3" 9}])
(to-dataset [[1 2 3]
[4 5 6]
[7 8 9]])
(conj-cols [1 4 7] [2 5 8] [3 6 9])
(conj-rows [1 2 3] [4 5 6] [7 8 9])
conj columns togetherCreating datasets](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-17-320.jpg)
![(use 'incanter.core)
(dataset ["x1" "x2" "x3"]
[[1 2 3]
[4 5 6]
[7 8 9]])
(to-dataset [{"x1" 1, "x2" 2, "x3" 3}
{"x1" 4, "x2" 5, "x3" 6}
{"x1" 7, "x2" 8, "x3" 9}])
(to-dataset [[1 2 3]
[4 5 6]
[7 8 9]])
(conj-cols [1 4 7] [2 5 8] [3 6 9])
(conj-rows [1 2 3] [4 5 6] [7 8 9])
conj rows togetherCreating datasets](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-18-320.jpg)
![(use '(incanter core datasets))
($ :speed (get-dataset :cars))
(4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15
15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24
25)
(with-data (get-dataset :cars)
[(mean ($ :speed))
(sd ($ :speed))])
(with-data (get-dataset :iris)
(view $data)
(view ($ [:Sepal.Length :Sepal.Width :Species])))
Column selection select a single column](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-26-320.jpg)
![(use '(incanter core datasets))
($ :speed (get-dataset :cars))
(4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15
15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24
25)
(with-data (get-dataset :cars)
[(mean ($ :speed))
(sd ($ :speed))])
[15.4 5.29]
(with-data (get-dataset :iris)
(view $data)
(view ($ [:Sepal.Length :Sepal.Width :Species])))
Column selection use with-data macro to bind dataset](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-27-320.jpg)
![(use '(incanter core datasets))
($ :speed (get-dataset :cars))
(4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15
15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24
25)
(with-data (get-dataset :cars)
[(mean ($ :speed))
(sd ($ :speed))])
[15.4 5.29]
(with-data (get-dataset :iris)
(view $data)
(view ($ [:Sepal.Length :Sepal.Width :Species])))
Column selection view dataset bound to $data](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-28-320.jpg)
![(use '(incanter core datasets))
($ :speed (get-dataset :cars))
(4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15
15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24
25)
(with-data (get-dataset :cars)
[(mean ($ :speed))
(sd ($ :speed))])
[15.4 5.29]
(with-data (get-dataset :iris)
(view $data)
(view ($ [:Sepal.Length :Sepal.Width :Species])))
Column selection select multiple columns](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-29-320.jpg)
![(use '(incanter core datasets))
($where {"Species" "setosa"}
(get-dataset :iris))
($where {"Petal.Width" {:lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}
"Species" {:in #{"virginica" "setosa"}}}
(get-dataset :iris))
($where (fn [row]
(or (< (row "Petal.Width") 1.0)
(> (row "Petal.Length") 5.0)))
(get-dataset :iris))
Row selection select rows where species equals ‘setosa’](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-30-320.jpg)
![(use '(incanter core datasets))
($where {"Species" "setosa"}
(get-dataset :iris))
($where {"Petal.Width" {:lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}
"Species" {:in #{"virginica" "setosa"}}}
(get-dataset :iris))
($where (fn [row]
(or (< (row "Petal.Width") 1.0)
(> (row "Petal.Length") 5.0)))
(get-dataset :iris))
Row selection select rows where petal-width < 1.5](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-31-320.jpg)
![(use '(incanter core datasets))
($where {"Species" "setosa"}
(get-dataset :iris))
($where {"Petal.Width" {:lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}
"Species" {:in #{"virginica" "setosa"}}}
(get-dataset :iris))
($where (fn [row]
(or (< (row "Petal.Width") 1.0)
(> (row "Petal.Length") 5.0)))
(get-dataset :iris))
Row selection select rows where 1.0 < petal-width < 1.5](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-32-320.jpg)
![(use '(incanter core datasets))
($where {"Species" "setosa"}
(get-dataset :iris))
($where {"Petal.Width" {:lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}
"Species" {:in #{"virginica" "setosa"}}}
(get-dataset :iris))
($where (fn [row]
(or (< (row "Petal.Width") 1.0)
(> (row "Petal.Length") 5.0)))
(get-dataset :iris))
Row selection select rows where species is ‘virginica’ or ‘setosa’](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-33-320.jpg)
![(use '(incanter core datasets))
($where {"Species" "setosa"}
(get-dataset :iris))
($where {"Petal.Width" {:lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}}
(get-dataset :iris))
($where {"Petal.Width" {:gt 1.0, :lt 1.5}
"Species" {:in #{"virginica" "setosa"}}}
(get-dataset :iris))
($where (fn [row]
(or (< (row "Petal.Width") 1.0)
(> (row "Petal.Length") 5.0)))
(get-dataset :iris))
Row selection select rows using an arbitrary predicate function](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-34-320.jpg)
![Sorting data hair & eye color data
(use '(incanter core datasets))
(with-data (get-dataset :hair-eye-color)
(view $data)
(view ($order :count :desc))
(view ($order [:hair :eye] :desc)))](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-35-320.jpg)
![Sorting data sort by count in descending order
(use '(incanter core datasets))
(with-data (get-dataset :hair-eye-color)
(view $data)
(view ($order :count :desc))
(view ($order [:hair :eye] :desc)))](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-36-320.jpg)
![Sorting data sort by hair and then eye color
(use '(incanter core datasets))
(with-data (get-dataset :hair-eye-color)
(view $data)
(view ($order :count :desc))
(view ($order [:hair :eye] :desc)))](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-37-320.jpg)
![Rolling up data mean petal-length by species
(use '(incanter core datasets stats))
(->> (get-dataset :iris)
($rollup mean :Petal.Length :Species)
view)
(->> (get-dataset :iris)
($rollup #(/ (sd %) (count %))
:Petal.Length :Species)
view)
(->> (get-dataset :hair-eye-color)
($rollup sum :count [:hair :eye])
($order :count :desc)
view)](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-38-320.jpg)
![Rolling up data standard error of petal-length
(use '(incanter core datasets stats))
(->> (get-dataset :iris)
($rollup mean :Petal.Length :Species)
view)
(->> (get-dataset :iris)
($rollup #(/ (sd %) (count %))
:Petal.Length :Species)
view)
(->> (get-dataset :hair-eye-color)
($rollup sum :count [:hair :eye])
($order :count :desc)
view)](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-39-320.jpg)
![Rolling up data sum of people with each hair/eye color combination
(use '(incanter core datasets stats))
(->> (get-dataset :iris)
($rollup mean :Petal.Length :Species)
view)
(->> (get-dataset :iris)
($rollup #(/ (sd %) (count %))
:Petal.Length :Species)
view)
(->> (get-dataset :hair-eye-color)
($rollup sum :count [:hair :eye])
($order :count :desc)
view)](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-40-320.jpg)
![Rolling up data sorted from most to least common
(use '(incanter core datasets stats))
(->> (get-dataset :iris)
($rollup mean :Petal.Length :Species)
view)
(->> (get-dataset :iris)
($rollup #(/ (sd %) (count %))
:Petal.Length :Species)
view)
(->> (get-dataset :hair-eye-color)
($rollup sum :count [:hair :eye])
($order :count :desc)
view)](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-41-320.jpg)
![(use '(incanter core charts datasets stats))
(with-data (get-dataset :iris)
(let [lm (linear-model ($ :Petal.Length) ($ :Petal.Width))]
(doto (scatter-plot :Petal.Width :Petal.Length
:theme :dark
:data ($where {"Petal.Length" {:lte 2.0}
"Petal.Width" {:lt 0.75}}))
(add-points :Petal.Width :Petal.Length
:data ($where {"Petal.Length" {:gt 2.0}
"Petal.Width" {:gte 0.75}}))
(add-lines :Petal.Width (:fitted lm))
view)))
add a regression lineAdding data](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-52-320.jpg)
![(with-data ($rollup :mean :count [:hair :eye]
(get-dataset :hair-eye-color))
(view $data)
(view (bar-chart :hair :count
:group-by :eye
:legend true
:theme :dark)))
Bar & line charts rollup the :count column using mean](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-55-320.jpg)
![(with-data ($rollup :mean :count [:hair :eye]
(get-dataset :hair-eye-color))
(view $data)
(view (bar-chart :hair :count
:group-by :eye
:legend true
:theme :dark)))
Bar & line charts bar-chart grouped by eye color](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-56-320.jpg)
![(with-data ($rollup :mean :count [:hair :eye]
(get-dataset :hair-eye-color))
(view $data)
(view (line-chart :hair :count
:group-by :eye
:legend true
:theme :dark)))
Bar & line charts line-chart grouped by eye color](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-57-320.jpg)
![(with-data (->> (get-dataset :hair-eye-color)
($where {:hair {:in #{"brown" "blond"}}})
($rollup :sum :count [:hair :eye])
($order :count :desc))
(view $data)
(view (bar-chart :hair :count
:group-by :eye
:legend true
:theme :dark)))
Bar & line charts sort by sum of :count column](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-58-320.jpg)
![(with-data (->> (get-dataset :hair-eye-color)
($where {:hair {:in #{"brown" "blond"}}})
($rollup :sum :count [:hair :eye])
($order :count :desc))
(view $data)
(view (bar-chart :hair :count
:group-by :eye
:legend true
:theme :dark)))
Bar & line charts bar-chart grouped by eye color](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-59-320.jpg)
![(with-data (->> (get-dataset :hair-eye-color)
($where {:hair {:in #{"brown" "blond"}}})
($rollup :sum :count [:hair :eye])
($order :count :desc))
(view $data)
(view (line-chart :hair :count
:group-by :eye
:legend true
:theme :dark)))
Bar & line charts line-chart grouped by eye color](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-60-320.jpg)
![function-plot of x3+2x2+2x+3XY & function plots
(use '(incanter (core charts optimize)))
(defn cubic [x] (+ (* x x x) (* 2 x x) (* 2 x) 3))
(doto (function-plot cubic -10 10 :theme :dark)
view)](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-62-320.jpg)
![add the derivative of the functionXY & function plots
(use '(incanter (core charts optimize)))
(defn cubic [x] (+ (* x x x) (* 2 x x) (* 2 x) 3))
(doto (function-plot cubic -10 10 :theme :dark)
(add-function (derivative cubic) -10 10)
view)](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-63-320.jpg)
![add a sine waveXY & function plots
(use '(incanter (core charts optimize)))
(defn cubic [x] (+ (* x x x) (* 2 x x) (* 2 x) 3))
(doto (function-plot cubic -10 10 :theme :dark)
(add-function (derivative cubic) -10 10)
(add-function #(* 1000 (sin %)) -10 10)
view)](https://image.slidesharecdn.com/incanterdata-sorcery-101024100110-phpapp02/85/Incanter-Data-Sorcery-64-320.jpg)
Ad
More Related Content
What's hot (20)
Similar to Incanter Data Sorcery (20)
Ad
More from elliando dias (20)
Ad
Recently uploaded (20)
Incanter Data Sorcery
- 2. David Edgar Liebke [email protected] Data Sorcery with Clojure & Incanter Introduction to Datasets & Charts
- 3. Datasets •Creating •Reading data •Saving data •Column selection •Row selection •Sorting •Rolling up Overview •What is Incanter? •Getting started •Incanter libraries Charts •Scatter plots •Chart options •Saving charts •Adding data •Bar & line charts •XY & function plots •Histograms & box-plots •Annotating charts Wrap up Outline
- 4. Datasets •Creating •Reading data •Saving data •Column selection •Row selection •Sorting •Rolling up Overview •What is Incanter? •Getting started •Incanter libraries Charts •Scatter plots •Chart options •Saving charts •Adding data •Bar & line charts •XY & function plots •Histograms & box-plots •Annotating charts Wrap up Outline
- 5. overviewWhat is Incanter? Incanter is a Clojure-based, R-like platform for statistical computing and graphics. It can be used either as a standalone, interactive data analysis environment or embedded within other analytics systems as a modular suite of libraries. Features •Charting & visualization functions •Data manipulation functions •Mathematical functions •Statistical functions •Matrix & linear algebra functions •Machine learning functions
- 6. who is Incanter?What is Incanter? Committers •David Edgar Liebke •Bradford Cross (FlightCaster) •Alex Ott (build master) •Tom Faulhaber (autodoc) •Sean Devlin (chrono 2.0) •Jared Strate (FlightCaster) Contributors •Chris Burroughs •Phil Hagelberg •Edmund Jackson (proposed time-series) •Steve Jenson •Steve Purcell •Brendan Ribera •Alexander Stoddard Community •Incanter Google group (108 members) •Github repository (197 watchers, 16 forks) Related project •Rincanter (Joel Boehland) As of 10 February 2010
- 7. Getting started incanter.org website
- 8. Getting started data-sorcery.org blog
- 9. quick startGetting started $ wget http://incanter.org/downloads/incanter-1.0-SNAPSHOT.zip $ unzip incanter-1.0-SNAPSHOT.zip $ cd incanter $ bin/clj user=> (use '(incanter core charts)) user=> (view (function-plot sin -4 4)))
- 10. •bayes •censored •charts (based on JFreeChart) •chrono (based on JodaTime) •classification •core (based on ParallelColt) •datasets •incremental-stats •information-theory •internal •io •mongodb (based on Congomongo) •optimize •pdf (based on iText) •probability •processing (based on Processing) •som •stats (based on ParallelColt) •transformations list of Incanter namespacesIncanter libraries
- 11. we will focus on charts & dataset functions in coreIncanter libraries •bayes •censored •charts •chrono •classification •core •datasets •incremental-stats •information-theory •internal •io •mongodb •optimize •pdf •probability •processing •som •stats •transformations
- 12. using functions from a few other namespacesIncanter libraries •bayes •censored •charts •chrono •classification •core •datasets •incremental-stats •information-theory •internal •io •mongodb •optimize •pdf •probability •processing •som •stats •transformations
- 13. Datasets •Creating •Reading data •Saving data •Column selection •Row selection •Sorting •Rolling up Overview •What is Incanter? •Getting started •Incanter libraries Charts •Scatter plots •Chart options •Saving charts •Adding data •Bar & line charts •XY & function plots •Histograms & box-plots •Annotating charts Wrap up Outline
- 14. (use 'incanter.core) (dataset ["x1" "x2" "x3"] [[1 2 3] [4 5 6] [7 8 9]]) (to-dataset [{"x1" 1, "x2" 2, "x3" 3} {"x1" 4, "x2" 5, "x3" 6} {"x1" 7, "x2" 8, "x3" 9}]) (to-dataset [[1 2 3] [4 5 6] [7 8 9]]) (conj-cols [1 4 7] [2 5 8] [3 6 9]) (conj-rows [1 2 3] [4 5 6] [7 8 9]) Creating datasets create a small dataset
- 15. (use 'incanter.core) (dataset ["x1" "x2" "x3"] [[1 2 3] [4 5 6] [7 8 9]]) (to-dataset [{"x1" 1, "x2" 2, "x3" 3} {"x1" 4, "x2" 5, "x3" 6} {"x1" 7, "x2" 8, "x3" 9}]) (to-dataset [[1 2 3] [4 5 6] [7 8 9]]) (conj-cols [1 4 7] [2 5 8] [3 6 9]) (conj-rows [1 2 3] [4 5 6] [7 8 9]) convert a sequence of mapsCreating datasets
- 16. (use 'incanter.core) (dataset ["x1" "x2" "x3"] [[1 2 3] [4 5 6] [7 8 9]]) (to-dataset [{"x1" 1, "x2" 2, "x3" 3} {"x1" 4, "x2" 5, "x3" 6} {"x1" 7, "x2" 8, "x3" 9}]) (to-dataset [[1 2 3] [4 5 6] [7 8 9]]) (conj-cols [1 4 7] [2 5 8] [3 6 9]) (conj-rows [1 2 3] [4 5 6] [7 8 9]) convert a sequence of sequencesCreating datasets
- 17. (use 'incanter.core) (dataset ["x1" "x2" "x3"] [[1 2 3] [4 5 6] [7 8 9]]) (to-dataset [{"x1" 1, "x2" 2, "x3" 3} {"x1" 4, "x2" 5, "x3" 6} {"x1" 7, "x2" 8, "x3" 9}]) (to-dataset [[1 2 3] [4 5 6] [7 8 9]]) (conj-cols [1 4 7] [2 5 8] [3 6 9]) (conj-rows [1 2 3] [4 5 6] [7 8 9]) conj columns togetherCreating datasets
- 18. (use 'incanter.core) (dataset ["x1" "x2" "x3"] [[1 2 3] [4 5 6] [7 8 9]]) (to-dataset [{"x1" 1, "x2" 2, "x3" 3} {"x1" 4, "x2" 5, "x3" 6} {"x1" 7, "x2" 8, "x3" 9}]) (to-dataset [[1 2 3] [4 5 6] [7 8 9]]) (conj-cols [1 4 7] [2 5 8] [3 6 9]) (conj-rows [1 2 3] [4 5 6] [7 8 9]) conj rows togetherCreating datasets
- 19. (use '(incanter core io)) (read-dataset "./data/cars.csv" :header true) (read-dataset "./data/cars.tdd" :header true :delim tab) (read-dataset "http://bit.ly/aZyjKa" :header true) (use 'incanter.datasets) (get-dataset :cars) (use 'incanter.mongodb) (use 'somnium.congomongo) (mongo! :db "mydb") (view (fetch-dataset :cars)) Reading data read a comma-delimited file
- 20. (use '(incanter core io)) (read-dataset "./data/cars.csv" :header true) (read-dataset "./data/cars.tdd" :header true :delim tab) (read-dataset "http://bit.ly/aZyjKa" :header true) (use 'incanter.datasets) (get-dataset :cars) (use 'incanter.mongodb) (use 'somnium.congomongo) (mongo! :db "mydb") (view (fetch-dataset :cars)) Reading data read a tab-delimited file
- 21. (use '(incanter core io)) (read-dataset "./data/cars.csv" :header true) (read-dataset "./data/cars.tdd" :header true :delim tab) (read-dataset "http://bit.ly/aZyjKa" :header true) (use 'incanter.datasets) (get-dataset :cars) (use 'incanter.mongodb) (use 'somnium.congomongo) (mongo! :db "mydb") (view (fetch-dataset :cars)) Reading data read a comma-delimited file from a URL
- 22. (use '(incanter core io)) (read-dataset "./data/cars.csv" :header true) (read-dataset "./data/cars.tdd" :header true :delim tab) (read-dataset "http://bit.ly/aZyjKa" :header true) (use 'incanter.datasets) (get-dataset :cars) (use 'incanter.mongodb) (use 'somnium.congomongo) (mongo! :db "mydb") (view (fetch-dataset :cars)) Reading data read a built-in sample dataset
- 23. (use '(incanter core io)) (read-dataset "./data/cars.csv" :header true) (read-dataset "./data/cars.tdd" :header true :delim tab) (read-dataset "http://bit.ly/aZyjKa" :header true) (use 'incanter.datasets) (get-dataset :cars) (use 'incanter.mongodb) (use 'somnium.congomongo) (mongo! :db "mydb") (view (fetch-dataset :cars)) Reading data retrieve a dataset from MongoDB
- 24. (use '(incanter core io)) (save data "./data.csv") (use '(incanter core mongodb)) (use 'somnium.congomongo) (mongo! :db "mydb") (insert-dataset :cars data) Saving data save a dataset in a CSV file
- 25. Saving data save a dataset in a MongoDB database (use '(incanter core io)) (save data "./data.csv") (use '(incanter core mongodb)) (use 'somnium.congomongo) (mongo! :db "mydb") (insert-dataset :cars data)
- 26. (use '(incanter core datasets)) ($ :speed (get-dataset :cars)) (4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15 15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24 25) (with-data (get-dataset :cars) [(mean ($ :speed)) (sd ($ :speed))]) (with-data (get-dataset :iris) (view $data) (view ($ [:Sepal.Length :Sepal.Width :Species]))) Column selection select a single column
- 27. (use '(incanter core datasets)) ($ :speed (get-dataset :cars)) (4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15 15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24 25) (with-data (get-dataset :cars) [(mean ($ :speed)) (sd ($ :speed))]) [15.4 5.29] (with-data (get-dataset :iris) (view $data) (view ($ [:Sepal.Length :Sepal.Width :Species]))) Column selection use with-data macro to bind dataset
- 28. (use '(incanter core datasets)) ($ :speed (get-dataset :cars)) (4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15 15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24 25) (with-data (get-dataset :cars) [(mean ($ :speed)) (sd ($ :speed))]) [15.4 5.29] (with-data (get-dataset :iris) (view $data) (view ($ [:Sepal.Length :Sepal.Width :Species]))) Column selection view dataset bound to $data
- 29. (use '(incanter core datasets)) ($ :speed (get-dataset :cars)) (4 4 7 7 8 9 10 10 10 11 11 12 12 12 12 13 13 13 13 14 14 14 14 15 15 15 16 16 17 17 17 18 18 18 18 19 19 19 20 20 20 20 20 22 23 24 24 24 24 25) (with-data (get-dataset :cars) [(mean ($ :speed)) (sd ($ :speed))]) [15.4 5.29] (with-data (get-dataset :iris) (view $data) (view ($ [:Sepal.Length :Sepal.Width :Species]))) Column selection select multiple columns
- 30. (use '(incanter core datasets)) ($where {"Species" "setosa"} (get-dataset :iris)) ($where {"Petal.Width" {:lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5} "Species" {:in #{"virginica" "setosa"}}} (get-dataset :iris)) ($where (fn [row] (or (< (row "Petal.Width") 1.0) (> (row "Petal.Length") 5.0))) (get-dataset :iris)) Row selection select rows where species equals ‘setosa’
- 31. (use '(incanter core datasets)) ($where {"Species" "setosa"} (get-dataset :iris)) ($where {"Petal.Width" {:lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5} "Species" {:in #{"virginica" "setosa"}}} (get-dataset :iris)) ($where (fn [row] (or (< (row "Petal.Width") 1.0) (> (row "Petal.Length") 5.0))) (get-dataset :iris)) Row selection select rows where petal-width < 1.5
- 32. (use '(incanter core datasets)) ($where {"Species" "setosa"} (get-dataset :iris)) ($where {"Petal.Width" {:lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5} "Species" {:in #{"virginica" "setosa"}}} (get-dataset :iris)) ($where (fn [row] (or (< (row "Petal.Width") 1.0) (> (row "Petal.Length") 5.0))) (get-dataset :iris)) Row selection select rows where 1.0 < petal-width < 1.5
- 33. (use '(incanter core datasets)) ($where {"Species" "setosa"} (get-dataset :iris)) ($where {"Petal.Width" {:lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5} "Species" {:in #{"virginica" "setosa"}}} (get-dataset :iris)) ($where (fn [row] (or (< (row "Petal.Width") 1.0) (> (row "Petal.Length") 5.0))) (get-dataset :iris)) Row selection select rows where species is ‘virginica’ or ‘setosa’
- 34. (use '(incanter core datasets)) ($where {"Species" "setosa"} (get-dataset :iris)) ($where {"Petal.Width" {:lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5}} (get-dataset :iris)) ($where {"Petal.Width" {:gt 1.0, :lt 1.5} "Species" {:in #{"virginica" "setosa"}}} (get-dataset :iris)) ($where (fn [row] (or (< (row "Petal.Width") 1.0) (> (row "Petal.Length") 5.0))) (get-dataset :iris)) Row selection select rows using an arbitrary predicate function
- 35. Sorting data hair & eye color data (use '(incanter core datasets)) (with-data (get-dataset :hair-eye-color) (view $data) (view ($order :count :desc)) (view ($order [:hair :eye] :desc)))
- 36. Sorting data sort by count in descending order (use '(incanter core datasets)) (with-data (get-dataset :hair-eye-color) (view $data) (view ($order :count :desc)) (view ($order [:hair :eye] :desc)))
- 37. Sorting data sort by hair and then eye color (use '(incanter core datasets)) (with-data (get-dataset :hair-eye-color) (view $data) (view ($order :count :desc)) (view ($order [:hair :eye] :desc)))
- 38. Rolling up data mean petal-length by species (use '(incanter core datasets stats)) (->> (get-dataset :iris) ($rollup mean :Petal.Length :Species) view) (->> (get-dataset :iris) ($rollup #(/ (sd %) (count %)) :Petal.Length :Species) view) (->> (get-dataset :hair-eye-color) ($rollup sum :count [:hair :eye]) ($order :count :desc) view)
- 39. Rolling up data standard error of petal-length (use '(incanter core datasets stats)) (->> (get-dataset :iris) ($rollup mean :Petal.Length :Species) view) (->> (get-dataset :iris) ($rollup #(/ (sd %) (count %)) :Petal.Length :Species) view) (->> (get-dataset :hair-eye-color) ($rollup sum :count [:hair :eye]) ($order :count :desc) view)
- 40. Rolling up data sum of people with each hair/eye color combination (use '(incanter core datasets stats)) (->> (get-dataset :iris) ($rollup mean :Petal.Length :Species) view) (->> (get-dataset :iris) ($rollup #(/ (sd %) (count %)) :Petal.Length :Species) view) (->> (get-dataset :hair-eye-color) ($rollup sum :count [:hair :eye]) ($order :count :desc) view)
- 41. Rolling up data sorted from most to least common (use '(incanter core datasets stats)) (->> (get-dataset :iris) ($rollup mean :Petal.Length :Species) view) (->> (get-dataset :iris) ($rollup #(/ (sd %) (count %)) :Petal.Length :Species) view) (->> (get-dataset :hair-eye-color) ($rollup sum :count [:hair :eye]) ($order :count :desc) view)
- 42. Datasets •Creating •Reading data •Saving data •Column selection •Row selection •Sorting •Rolling up Overview •What is Incanter? •Getting started •Incanter libraries Charts •Scatter plots •Chart options •Saving charts •Adding data •Bar & line charts •XY & function plots •Histograms & box-plots •Annotating charts Wrap up Outline
- 43. (view (scatter-plot :Sepal.Length :Sepal.Width :data (get-dataset :iris))) Scatter plots plot ‘sepal length’ vs ‘sepal width’
- 44. (view (scatter-plot :Sepal.Length :Sepal.Width :data (get-dataset :iris) :theme :dark)) use dark chart themeChart options
- 45. (view (scatter-plot :Sepal.Length :Sepal.Width :data (get-dataset :iris) :theme :dark :group-by :Species)) group by species valuesChart options
- 46. (view (scatter-plot :Sepal.Length :Sepal.Width :data (get-dataset :iris) :theme :dark :group-by :Species :title "Fisher Iris Data" :x-label "Sepal Length (cm)" :y-label "Sepal Width (cm)")) change chart title & axes labelsChart options
- 47. (save (scatter-plot :Sepal.Length :Sepal.Width :data (get-dataset :iris) :theme :dark :group-by :Species :title "Fisher Iris Data" :x-label "Sepal Length (cm)" :y-label "Sepal Width (cm)") "./iris-plot.png") Saving charts save chart to a PNG file
- 48. (def output-stream (java.io.ByteArrayOutputStream.)) (save (scatter-plot :Sepal.Length :Sepal.Width :data (get-dataset :iris) :theme :dark :group-by :Species :title "Fisher Iris Data" :x-label "Sepal Length (cm)" :y-label "Sepal Width (cm)") output-stream) save chart to an OutputStreamSaving charts
- 49. (use 'incanter.pdf) (save-pdf (scatter-plot :Sepal.Length :Sepal.Width :data (get-dataset :iris) :theme :dark :group-by :Species :title "Fisher Iris Data" :x-label "Sepal Length (cm)" :y-label "Sepal Width (cm)") "./iris-plot.pdf") save chart to a PDF fileSaving charts
- 50. (use '(incanter core charts datasets)) (with-data (get-dataset :iris) (doto (scatter-plot :Petal.Width :Petal.Length :theme :dark :data ($where {"Petal.Length" {:lte 2.0} "Petal.Width" {:lt 0.75}})) view))) Adding data plot points where petal-length <= 2 & petal-width < .75
- 51. (use '(incanter core charts datasets)) (with-data (get-dataset :iris) (doto (scatter-plot :Petal.Width :Petal.Length :theme :dark :data ($where {"Petal.Length" {:lte 2.0} "Petal.Width" {:lt 0.75}})) (add-points :Petal.Width :Petal.Length :data ($where {"Petal.Length" {:gt 2.0} "Petal.Width" {:gte 0.75}})) view))) add points where petal-length > 2 & petal-width >= .75Adding data
- 52. (use '(incanter core charts datasets stats)) (with-data (get-dataset :iris) (let [lm (linear-model ($ :Petal.Length) ($ :Petal.Width))] (doto (scatter-plot :Petal.Width :Petal.Length :theme :dark :data ($where {"Petal.Length" {:lte 2.0} "Petal.Width" {:lt 0.75}})) (add-points :Petal.Width :Petal.Length :data ($where {"Petal.Length" {:gt 2.0} "Petal.Width" {:gte 0.75}})) (add-lines :Petal.Width (:fitted lm)) view))) add a regression lineAdding data
- 53. (use '(incanter core charts datasets)) (with-data ($rollup mean :Sepal.Length :Species (get-dataset :iris)) (view (bar-chart :Species :Sepal.Length :theme :dark))) Bar & line charts bar-chart of mean sepal-length for each species
- 54. (use '(incanter core charts datasets)) (with-data ($rollup mean :Sepal.Length :Species (get-dataset :iris)) (view (line-chart :Species :Sepal.Length :theme :dark))) Bar & line charts line-chart of mean sepal-length for each species
- 55. (with-data ($rollup :mean :count [:hair :eye] (get-dataset :hair-eye-color)) (view $data) (view (bar-chart :hair :count :group-by :eye :legend true :theme :dark))) Bar & line charts rollup the :count column using mean
- 56. (with-data ($rollup :mean :count [:hair :eye] (get-dataset :hair-eye-color)) (view $data) (view (bar-chart :hair :count :group-by :eye :legend true :theme :dark))) Bar & line charts bar-chart grouped by eye color
- 57. (with-data ($rollup :mean :count [:hair :eye] (get-dataset :hair-eye-color)) (view $data) (view (line-chart :hair :count :group-by :eye :legend true :theme :dark))) Bar & line charts line-chart grouped by eye color
- 58. (with-data (->> (get-dataset :hair-eye-color) ($where {:hair {:in #{"brown" "blond"}}}) ($rollup :sum :count [:hair :eye]) ($order :count :desc)) (view $data) (view (bar-chart :hair :count :group-by :eye :legend true :theme :dark))) Bar & line charts sort by sum of :count column
- 59. (with-data (->> (get-dataset :hair-eye-color) ($where {:hair {:in #{"brown" "blond"}}}) ($rollup :sum :count [:hair :eye]) ($order :count :desc)) (view $data) (view (bar-chart :hair :count :group-by :eye :legend true :theme :dark))) Bar & line charts bar-chart grouped by eye color
- 60. (with-data (->> (get-dataset :hair-eye-color) ($where {:hair {:in #{"brown" "blond"}}}) ($rollup :sum :count [:hair :eye]) ($order :count :desc)) (view $data) (view (line-chart :hair :count :group-by :eye :legend true :theme :dark))) Bar & line charts line-chart grouped by eye color
- 61. xy-plot of two continuous variablesXY & function plots (use '(incanter (core charts))) (with-data (get-dataset :cars) (view (xy-plot :speed :dist :theme :dark)))
- 62. function-plot of x3+2x2+2x+3XY & function plots (use '(incanter (core charts optimize))) (defn cubic [x] (+ (* x x x) (* 2 x x) (* 2 x) 3)) (doto (function-plot cubic -10 10 :theme :dark) view)
- 63. add the derivative of the functionXY & function plots (use '(incanter (core charts optimize))) (defn cubic [x] (+ (* x x x) (* 2 x x) (* 2 x) 3)) (doto (function-plot cubic -10 10 :theme :dark) (add-function (derivative cubic) -10 10) view)
- 64. add a sine waveXY & function plots (use '(incanter (core charts optimize))) (defn cubic [x] (+ (* x x x) (* 2 x x) (* 2 x) 3)) (doto (function-plot cubic -10 10 :theme :dark) (add-function (derivative cubic) -10 10) (add-function #(* 1000 (sin %)) -10 10) view)
- 65. plot a sample from a gamma distributionHistograms & box-plots (use '(incanter (core charts stats))) (doto (histogram (sample-gamma 1000) :density true :nbins 30 :theme :dark) view)
- 66. add a gamma pdf lineHistograms & box-plots (use '(incanter (core charts stats))) (doto (histogram (sample-gamma 1000) :density true :nbins 30 :theme :dark) (add-function pdf-gamma 0 8) view)
- 67. box-plots of petal-width grouped by speciesHistograms & box-plots (use '(incanter core datasets)) (with-data (get-dataset :iris) (view (box-plot :Petal.Width :group-by :Species :theme :dark)))
- 68. plot sin waveAnnotating charts (use '(incanter core charts)) (doto (function-plot sin -10 10) view)
- 69. add text annotation (black text only)Annotating charts (use '(incanter core charts)) (doto (function-plot sin -10 10) (add-text 0 0 "text at (0,0)") view)
- 70. add pointer annotationsAnnotating charts (use '(incanter core charts)) (doto (function-plot sin -10 10) (add-text 0 0 "text at (0,0)") (add-pointer (- Math/PI) (sin (- Math/PI)) :text "pointer at (sin -pi)") (add-pointer Math/PI (sin Math/PI) :text "pointer at(sin pi)" :angle :ne) (add-pointer (* 1/2 Math/PI) (sin (* 1/2 Math/PI)) :text "pointer at(sin pi/2)" :angle :south) view)
- 71. Datasets •Creating •Reading data •Saving data •Column selection •Row selection •Sorting •Rolling up Overview •What is Incanter? •Getting started •Incanter libraries Charts •Scatter plots •Chart options •Saving charts •Adding data •Bar & line charts •XY & function plots •Histograms & box-plots •Annotating charts Wrap up Outline
- 72. learn more and contributeWrap up Learn more •Visit http://incanter.org •Visit http://data-sorcery.org Join the community and contribute •Join the Google group: http://groups.google.com/group/incanter •Follow Incanter on Github: http://github.com/liebke/incanter •Follow @liebke on Twitter
- 73. Thank you