SlideShare a Scribd company logo
GETTING STARTED WITH
ELASTICSEARCH ON WINDOWS
AND .NET WITH NEST
A short introduction

Oslo/NNUG Meetup
Tomas Jansson
29/01/2014
THIS IS ME

Tomas Jansson
Manager & Group Lead .NET
BEKK Oslo
@TomasJansson
tomas.jansson@bekk.no
github.com/mastoj
blog.tomasjansson.com
TL;DR;

https://github.com/mastoj/NestDemo
Getting started with Elasticsearch and .NET
Getting started with Elasticsearch and .NET
AUDIENCE

N00b

Expert

N00b

Expert
BACKGROUND

This is the data and we need this new application
THE MASTERPLAN
WHAT I WANT TO SHOW YOU IS...

Elasticsearch is awesome
Indexing using NEST

Querying using NEST
... not about advanced elasticsearch hosting
Getting started with Elasticsearch and .NET
INSTALLATION

Great news, install as a service added in 0.90.5

Powershell to
the rescue
NEST

Abstraction
over
Elasticsearch

There is an low level abstraction as well called RawElasticClient
NEST

Abstraction
Fluent &
Strongly
over
Elasticsearch
typed
Functional C#
FUNC DEMO

C:Devgit> scriptcs
scriptcs (ctrl-c or blank to exit)

> Func<int, int, int> add = (x, y) => x + y;
> add(1, 3)
4

Func  executable
SIMPLE EXPRESSION DEMO

> using System.Linq.Expressions;
> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;
> addExpr(1, 3)

Expression  ”function description”

(1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method.
> var binExpr = addExpr.Body as BinaryExpression;
> Console.WriteLine(binExpr);
(x + y)
> var add2 = addExpr.Compile();
> add2(3, 1);
4
MORE COMPLEX EXPRESSION DEMO

> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString
MORE COMPLEX EXPRESSION DEMO

> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
Enables us to translate from one domain to another in an ”easy” manner
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString
Show me the
code!
ELASTICSEARCH CONNECTION

public class ElasticClientWrapper : ElasticClient
{
private static string _connectionString = Settings.ElasticSearchServer;
private static ConnectionSettings _settings =
new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200
.SetDefaultIndex(Settings.Alias) //"customer_product_mapping"
.UsePrettyResponses();
public ElasticClientWrapper()
: base(_settings)
{
}
}
//usage
var client = new ElasticClientWrapper();
MAPPING

public class Product
{
public double UnitPrice { get; set; }
public int TotalQuantity { get; set; }
[ElasticProperty(Index = FieldIndexOption.not_analyzed)]
public string ProductName { get; set; }
[ElasticProperty(Index = FieldIndexOption.not_analyzed)]
public string CategoryName { get; set; }
}
public class Customer
{
public string CustomerID { get; set; }
public string CompanyName { get; set; }
public string Address { get; set; }
public string City { get; set; }
public string Country { get; set; }
[ElasticProperty(Type = FieldType.nested)]
public Product[] Products { get; set; }
}
MAPPING & INDEXING

_client = new ElasticClientWrapper();
Mapping created from attributes
_client.CreateIndex("indexName", s =>
s.AddMapping<Customer>(m => m.MapFromAttributes()));

var customers = _customerRepo.GetCustomers();
_client.IndexMany(customers, "indexName");

Indexing will use the mapping
for the specified index
There is async versions of the
methods
ALIAS

_client = new ElasticClientWrapper();
_client.Alias("indexName", "aliasName");

Alias

Index_01
SWAPPING

_client = new ElasticClientWrapper();
_client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } );

Alias
1.

Index_02

Swap

3.

Index_01

Create new index

2.

Alias

Delete old index
MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES)

public class SearchModel
{
private int? _numberToTake;
public string Query { get; set; }
public Dictionary<string, IEnumerable<string>> Filter { get; set; }
public int? NumberToTake
{
get { return _numberToTake.HasValue ? _numberToTake.Value : 25; }
set { _numberToTake = value; }
}

}
QUERYING

Elasticsearch

NEST

{

_client.Search<Customer>(sd => sd
.QueryString(Input.Query));

"query": {
"query_string": {
"query": "tomas"
}
}
}
FUZZY

Elasticsearch

NEST

{

_client.Search<Customer>(sd => sd
.Query(q => q
.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query))));

"query": {
"fuzzy": {
"_all": {
"min_similarity": 0.6,
"prefix_length": 1,
"value": "tomas"
}
}
}
}

Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»
FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH

{
"query": {
"bool": {
"should": [{
"match": {
"_all": {
"query": "tomas"
}
}
},
{
"fuzzy": {
"_all": {
"boost": 0.1,
"min_similarity": 0.6,
"prefix_length": 1,
"value": "tomas"
}
}
}]
}
}
}
FUZZY IMPROVED (USING BOOL QUERY) - NEST

_client.Search<Customer>(sd => sd
.Query(q => q
.Bool(b => b
.Should(new Func<QueryDescriptor<Customer>, BaseQuery>[]
{
_ => _.Match(m => m
.OnField("_all")
.QueryString(Input.Query)),
_ => _.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query)
.Boost(0.1))
}))));
HIGHLIGHT RESULT - ELASTICSEARCH

{
"query": {
// see previous example
},
"highlight": {
"pre_tags": [
"<span class='highlight'>"
],
"post_tags": [
"</span>"
],
"fields": {
"companyName": {
"fragment_size": 100,
"number_of_fragments": 1
}
}
}

}
HIGHLIGHT RESULT - NEST

_client.Search<Customer>(sd => sd
.Query( /* See previous example */ )
.Highlight(h => h
.PreTags("<span class='highlight'>")
.PostTags("</span>")
.OnFields(new Action<HighlightFieldDescriptor<Customer>>[]
{
_ => _
.OnField(c => c.CompanyName)
.NumberOfFragments(1)
.FragmentSize(100)
})));
FACETS - ELASTICSEARCH

{
"query": { /* See previous example */ },
"highlight": { /* See previous example */ },
"facets": {
"products.productName": {
"nested": "products",
"terms": { "field": "products.productName", "size": 1000 }
},
"products.categoryName": {
"nested": "products",
"terms": { "field": "products.categoryName", "size": 1000 }
},
"country": {
"terms": { "field": "country", "size": 1000 }
}
}
}
FACETS - NEST

_client.Search<Customer>(sd => sd
.Query( /* See previous example */ )
.Highlight( /* See previous example */ )
.FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].ProductName)
.Size(1000))
.FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].CategoryName)
.Size(1000))
.FacetTerm(f => f
.OnField(c => c.Country)
.Size(1000)));
http://go-gaga-over-testing.blogspot.no/2011/09/solution-to-warning-in-quality-center.html
FILTERS - ELASTICSEARCH
{
"query": {
"filtered": {
"query": { /* See previous example */ },
"filter": {
"bool": {
"must": [
{
"terms": { "country": ["usa"] }
},
{
"nested": {
"query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } },
"path": "products"
}
},
{
"nested": {
"query": { "terms": { "products.productName": ["Chai"] } },
"path": "products"
}
}
]
}
}
}
},
"facets": { /* See previous example */},
"highlight": { /* See previous example */ }
}
FILTERS – NEST – PART 1, THE CUSTOMERS FILTER

private static BaseFilter AddCustomerFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Terms(propExpr, items.ToArray());
}
FILTERS – NEST – PART 1, THE PRODUCTS FILTER

private static BaseFilter AddProductsFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));
}
FILTERS – NEST – PART 1, THE MAGIC DICTIONARY

public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =
new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>()
{
{"products.productName", ps => AddProductsFilter(ps, c => c
.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c
.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}
};
FILTERS – NEST – PART 1, ALL THE HELPERS

private static BaseFilter AddCustomerFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Terms(propExpr, items.ToArray());
}
private static BaseFilter AddProductsFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));
}
public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =
new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>()
{
{"products.productName", ps => AddProductsFilter(ps, c => c
.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c
.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}
};
FILTERS – NEST – PART 2, THE QUERY
_client.Search<Customer>(sd => sd
.Query(q => q
.Filtered(fq =>
{
fq.Query(qs =>
{
if (!string.IsNullOrEmpty(Input.Query))
{
qs.Bool( /* See previous example */ ));
}
else
{
qs.MatchAll();
}
return qs;
});
if (Input.Filter.Count > 0)
{
var filters =
Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray();
fq.Filter(fs => fs.Bool(bf => bf.Must(filters)));
}
}))
.Highlight( /* See previous example */ )
.FacetTerm( /* See previous example */ )
.FacetTerm( /* See previous example */ )
.FacetTerm( /* See previous example */ );
SUMMARY

Elasticsearch

NEST

Easy installation

Strongly typed client

Awesome search engine

Fluent
Abstraction over Elasticsearch
RESOURCES

Demo code: https://github.com/mastoj/NestDemo
Nest documentation: http://nest.azurewebsites.net/
Nest source code: https://github.com/Mpdreamz/NEST
Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net
Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense
Questions?
Thank you!
@TomasJansson

More Related Content

What's hot (20)

PDF
Postgresql search demystified
javier ramirez
 
PDF
MongoDB World 2016: Deciphering .explain() Output
MongoDB
 
PPTX
Morphia: Simplifying Persistence for Java and MongoDB
Jeff Yemin
 
PPTX
Webinar: Index Tuning and Evaluation
MongoDB
 
PPTX
Cassandra 2.2 & 3.0
Victor Coustenoble
 
PDF
Java Persistence Frameworks for MongoDB
MongoDB
 
PPT
Developing application for Windows Phone 7 in TDD
Michele Capra
 
PDF
concurrency with GPars
Paul King
 
PPTX
GreenDao Introduction
Booch Lin
 
PPTX
Simplifying Persistence for Java and MongoDB with Morphia
MongoDB
 
PDF
Search Evolution - Von Lucene zu Solr und ElasticSearch
Florian Hopf
 
PPTX
Solr 6 Feature Preview
Yonik Seeley
 
PDF
Webinar: MongoDB Persistence with Java and Morphia
MongoDB
 
PDF
Requery overview
Sunghyouk Bae
 
PDF
Indexing and Query Optimizer (Mongo Austin)
MongoDB
 
PPT
Fast querying indexing for performance (4)
MongoDB
 
PDF
Green dao
彥彬 洪
 
PDF
Learn Ajax here
jarnail
 
PDF
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자
Donghyeok Kang
 
PDF
Scala ActiveRecord
scalaconfjp
 
Postgresql search demystified
javier ramirez
 
MongoDB World 2016: Deciphering .explain() Output
MongoDB
 
Morphia: Simplifying Persistence for Java and MongoDB
Jeff Yemin
 
Webinar: Index Tuning and Evaluation
MongoDB
 
Cassandra 2.2 & 3.0
Victor Coustenoble
 
Java Persistence Frameworks for MongoDB
MongoDB
 
Developing application for Windows Phone 7 in TDD
Michele Capra
 
concurrency with GPars
Paul King
 
GreenDao Introduction
Booch Lin
 
Simplifying Persistence for Java and MongoDB with Morphia
MongoDB
 
Search Evolution - Von Lucene zu Solr und ElasticSearch
Florian Hopf
 
Solr 6 Feature Preview
Yonik Seeley
 
Webinar: MongoDB Persistence with Java and Morphia
MongoDB
 
Requery overview
Sunghyouk Bae
 
Indexing and Query Optimizer (Mongo Austin)
MongoDB
 
Fast querying indexing for performance (4)
MongoDB
 
Green dao
彥彬 洪
 
Learn Ajax here
jarnail
 
[제1회 루씬 한글분석기 기술세미나] solr로 나만의 검색엔진을 만들어보자
Donghyeok Kang
 
Scala ActiveRecord
scalaconfjp
 

Viewers also liked (14)

PPTX
Battle of the giants: Apache Solr vs ElasticSearch
Rafał Kuć
 
PDF
Elasticsearch in 15 minutes
David Pilato
 
PPT
Elastic search introduction
Jackson dos Santos Olveira
 
PDF
Catalogue de stage
mzoughi Anis
 
PDF
Advancing Earth Science with Elasticsearch at Terradue
terradue
 
PPTX
Scaling Solr with Solr Cloud
Sematext Group, Inc.
 
PDF
Time Series Processing with Apache Spark
QAware GmbH
 
PPTX
Introduction to Elasticsearch with basics of Lucene
Rahul Jain
 
PPT
Introduction to Apache Solr.
ashish0x90
 
PDF
Logging with Elasticsearch, Logstash & Kibana
Amazee Labs
 
PPT
Boosting Documents in Solr by Recency, Popularity, and User Preferences
Lucidworks (Archived)
 
PDF
Apache Solr crash course
Tommaso Teofili
 
PPT
Running High Performance and Fault Tolerant Elasticsearch Clusters on Docker
Sematext Group, Inc.
 
Battle of the giants: Apache Solr vs ElasticSearch
Rafał Kuć
 
Elasticsearch in 15 minutes
David Pilato
 
Elastic search introduction
Jackson dos Santos Olveira
 
Catalogue de stage
mzoughi Anis
 
Advancing Earth Science with Elasticsearch at Terradue
terradue
 
Scaling Solr with Solr Cloud
Sematext Group, Inc.
 
Time Series Processing with Apache Spark
QAware GmbH
 
Introduction to Elasticsearch with basics of Lucene
Rahul Jain
 
Introduction to Apache Solr.
ashish0x90
 
Logging with Elasticsearch, Logstash & Kibana
Amazee Labs
 
Boosting Documents in Solr by Recency, Popularity, and User Preferences
Lucidworks (Archived)
 
Apache Solr crash course
Tommaso Teofili
 
Running High Performance and Fault Tolerant Elasticsearch Clusters on Docker
Sematext Group, Inc.
 
Ad

Similar to Getting started with Elasticsearch and .NET (20)

PPTX
“Elasticsearch for .NET developers”
HYS Enterprise
 
PPTX
Getting Started With Elasticsearch In .NET
Ahmed Abd Ellatif
 
PPTX
Getting started with Elasticsearch in .net
Ismaeel Enjreny
 
PDF
Introduction to Elasticsearch
Sperasoft
 
PDF
Elasto Mania
andrefsantos
 
PDF
Enhancement of Searching and Analyzing the Document using Elastic Search
IRJET Journal
 
PDF
Faster and better search results with Elasticsearch
Enrico Polesel
 
PPSX
Elasticsearch - basics and beyond
Ernesto Reig
 
PDF
Elasticsearch speed is key
Enterprise Search Warsaw Meetup
 
PDF
Fazendo mágica com ElasticSearch
Pedro Franceschi
 
PPTX
TechEd AU 2014: Microsoft Azure DocumentDB Deep Dive
Intergen
 
PDF
elasticsearch - advanced features in practice
Jano Suchal
 
PPTX
Elasticsearch as a search alternative to a relational database
Kristijan Duvnjak
 
PDF
JavaCro'15 - Elasticsearch as a search alternative to a relational database -...
HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
 
PDF
Elasticsearch
Amine Ferchichi
 
PDF
Data modeling for Elasticsearch
Florian Hopf
 
PPTX
Elasticsearch an overview
Amit Juneja
 
PPTX
Examiness hints and tips from the trenches
Ismail Mayat
 
PDF
Elasticsearch first-steps
Matteo Moci
 
PPTX
Cloud powered search
Codecamp Romania
 
“Elasticsearch for .NET developers”
HYS Enterprise
 
Getting Started With Elasticsearch In .NET
Ahmed Abd Ellatif
 
Getting started with Elasticsearch in .net
Ismaeel Enjreny
 
Introduction to Elasticsearch
Sperasoft
 
Elasto Mania
andrefsantos
 
Enhancement of Searching and Analyzing the Document using Elastic Search
IRJET Journal
 
Faster and better search results with Elasticsearch
Enrico Polesel
 
Elasticsearch - basics and beyond
Ernesto Reig
 
Elasticsearch speed is key
Enterprise Search Warsaw Meetup
 
Fazendo mágica com ElasticSearch
Pedro Franceschi
 
TechEd AU 2014: Microsoft Azure DocumentDB Deep Dive
Intergen
 
elasticsearch - advanced features in practice
Jano Suchal
 
Elasticsearch as a search alternative to a relational database
Kristijan Duvnjak
 
JavaCro'15 - Elasticsearch as a search alternative to a relational database -...
HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
 
Elasticsearch
Amine Ferchichi
 
Data modeling for Elasticsearch
Florian Hopf
 
Elasticsearch an overview
Amit Juneja
 
Examiness hints and tips from the trenches
Ismail Mayat
 
Elasticsearch first-steps
Matteo Moci
 
Cloud powered search
Codecamp Romania
 
Ad

More from Tomas Jansson (12)

PPTX
Functional webapplicaations using fsharp and suave
Tomas Jansson
 
PPTX
F# as our day job by 2016
Tomas Jansson
 
PPTX
What does the future hold for us in asp.net 5
Tomas Jansson
 
PPTX
Polyglot heaven
Tomas Jansson
 
PPTX
OWIN Web API with Linky
Tomas Jansson
 
PPTX
Roslyn
Tomas Jansson
 
PPTX
File -> new project to deploy in 10 minutes with TeamCity and Octopus Deploy
Tomas Jansson
 
PPTX
Deployment taken seriously with Octopus Deploy and TeamCity
Tomas Jansson
 
PPTX
State or intent
Tomas Jansson
 
PPTX
NServiceBus workshop presentation
Tomas Jansson
 
PPTX
SignalR - Building an async web app with .NET
Tomas Jansson
 
PPTX
REST for .NET - Introduction to ASP.NET Web API
Tomas Jansson
 
Functional webapplicaations using fsharp and suave
Tomas Jansson
 
F# as our day job by 2016
Tomas Jansson
 
What does the future hold for us in asp.net 5
Tomas Jansson
 
Polyglot heaven
Tomas Jansson
 
OWIN Web API with Linky
Tomas Jansson
 
File -> new project to deploy in 10 minutes with TeamCity and Octopus Deploy
Tomas Jansson
 
Deployment taken seriously with Octopus Deploy and TeamCity
Tomas Jansson
 
State or intent
Tomas Jansson
 
NServiceBus workshop presentation
Tomas Jansson
 
SignalR - Building an async web app with .NET
Tomas Jansson
 
REST for .NET - Introduction to ASP.NET Web API
Tomas Jansson
 

Recently uploaded (20)

PDF
NLJUG Speaker academy 2025 - first session
Bert Jan Schrijver
 
PDF
Transforming Utility Networks: Large-scale Data Migrations with FME
Safe Software
 
PDF
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
PDF
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
Edge AI and Vision Alliance
 
PDF
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PPTX
Agentforce World Tour Toronto '25 - Supercharge MuleSoft Development with Mod...
Alexandra N. Martinez
 
PDF
Mastering Financial Management in Direct Selling
Epixel MLM Software
 
PPTX
Mastering ODC + Okta Configuration - Chennai OSUG
HathiMaryA
 
PPTX
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
PPTX
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
PDF
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
PDF
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
PPT
Ericsson LTE presentation SEMINAR 2010.ppt
npat3
 
PPTX
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
PDF
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
PPTX
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
PDF
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
PDF
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
PDF
NASA A Researcher’s Guide to International Space Station : Physical Sciences ...
Dr. PANKAJ DHUSSA
 
NLJUG Speaker academy 2025 - first session
Bert Jan Schrijver
 
Transforming Utility Networks: Large-scale Data Migrations with FME
Safe Software
 
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
Edge AI and Vision Alliance
 
Peak of Data & AI Encore AI-Enhanced Workflows for the Real World
Safe Software
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Agentforce World Tour Toronto '25 - Supercharge MuleSoft Development with Mod...
Alexandra N. Martinez
 
Mastering Financial Management in Direct Selling
Epixel MLM Software
 
Mastering ODC + Okta Configuration - Chennai OSUG
HathiMaryA
 
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
“Computer Vision at Sea: Automated Fish Tracking for Sustainable Fishing,” a ...
Edge AI and Vision Alliance
 
Ericsson LTE presentation SEMINAR 2010.ppt
npat3
 
MuleSoft MCP Support (Model Context Protocol) and Use Case Demo
shyamraj55
 
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
Future Tech Innovations 2025 – A TechLists Insight
TechLists
 
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
Reverse Engineering of Security Products: Developing an Advanced Microsoft De...
nwbxhhcyjv
 
NASA A Researcher’s Guide to International Space Station : Physical Sciences ...
Dr. PANKAJ DHUSSA
 

Getting started with Elasticsearch and .NET

  • 1. GETTING STARTED WITH ELASTICSEARCH ON WINDOWS AND .NET WITH NEST A short introduction Oslo/NNUG Meetup Tomas Jansson 29/01/2014
  • 2. THIS IS ME Tomas Jansson Manager & Group Lead .NET BEKK Oslo @TomasJansson [email protected] github.com/mastoj blog.tomasjansson.com
  • 7. BACKGROUND This is the data and we need this new application
  • 9. WHAT I WANT TO SHOW YOU IS... Elasticsearch is awesome Indexing using NEST Querying using NEST ... not about advanced elasticsearch hosting
  • 11. INSTALLATION Great news, install as a service added in 0.90.5 Powershell to the rescue
  • 12. NEST Abstraction over Elasticsearch There is an low level abstraction as well called RawElasticClient
  • 15. FUNC DEMO C:Devgit> scriptcs scriptcs (ctrl-c or blank to exit) > Func<int, int, int> add = (x, y) => x + y; > add(1, 3) 4 Func  executable
  • 16. SIMPLE EXPRESSION DEMO > using System.Linq.Expressions; > Expression<Func<int, int, int>> addExpr = (x, y) => x + y; > addExpr(1, 3) Expression  ”function description” (1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method. > var binExpr = addExpr.Body as BinaryExpression; > Console.WriteLine(binExpr); (x + y) > var add2 = addExpr.Compile(); > add2(3, 1); 4
  • 17. MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString
  • 18. MORE COMPLEX EXPRESSION DEMO > public class SomeClass { public string MyString { get; set; } } > Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString; > var compExpr = propExpr.Compile(); > var obj = new SomeClass { MyString = "Hello world" }; > compExpr(obj) Hello worldHello world > var body = propExpr.Body as BinaryExpression; Enables us to translate from one domain to another in an ”easy” manner > Console.WriteLine(body); (y.MyString + y.MyString) > var left = body.Left as MemberExpression; > Console.WriteLine(left.Member.Name); MyString
  • 20. ELASTICSEARCH CONNECTION public class ElasticClientWrapper : ElasticClient { private static string _connectionString = Settings.ElasticSearchServer; private static ConnectionSettings _settings = new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200 .SetDefaultIndex(Settings.Alias) //"customer_product_mapping" .UsePrettyResponses(); public ElasticClientWrapper() : base(_settings) { } } //usage var client = new ElasticClientWrapper();
  • 21. MAPPING public class Product { public double UnitPrice { get; set; } public int TotalQuantity { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string ProductName { get; set; } [ElasticProperty(Index = FieldIndexOption.not_analyzed)] public string CategoryName { get; set; } } public class Customer { public string CustomerID { get; set; } public string CompanyName { get; set; } public string Address { get; set; } public string City { get; set; } public string Country { get; set; } [ElasticProperty(Type = FieldType.nested)] public Product[] Products { get; set; } }
  • 22. MAPPING & INDEXING _client = new ElasticClientWrapper(); Mapping created from attributes _client.CreateIndex("indexName", s => s.AddMapping<Customer>(m => m.MapFromAttributes())); var customers = _customerRepo.GetCustomers(); _client.IndexMany(customers, "indexName"); Indexing will use the mapping for the specified index There is async versions of the methods
  • 23. ALIAS _client = new ElasticClientWrapper(); _client.Alias("indexName", "aliasName"); Alias Index_01
  • 24. SWAPPING _client = new ElasticClientWrapper(); _client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } ); Alias 1. Index_02 Swap 3. Index_01 Create new index 2. Alias Delete old index
  • 25. MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES) public class SearchModel { private int? _numberToTake; public string Query { get; set; } public Dictionary<string, IEnumerable<string>> Filter { get; set; } public int? NumberToTake { get { return _numberToTake.HasValue ? _numberToTake.Value : 25; } set { _numberToTake = value; } } }
  • 27. FUZZY Elasticsearch NEST { _client.Search<Customer>(sd => sd .Query(q => q .Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query)))); "query": { "fuzzy": { "_all": { "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } } } Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»
  • 28. FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH { "query": { "bool": { "should": [{ "match": { "_all": { "query": "tomas" } } }, { "fuzzy": { "_all": { "boost": 0.1, "min_similarity": 0.6, "prefix_length": 1, "value": "tomas" } } }] } } }
  • 29. FUZZY IMPROVED (USING BOOL QUERY) - NEST _client.Search<Customer>(sd => sd .Query(q => q .Bool(b => b .Should(new Func<QueryDescriptor<Customer>, BaseQuery>[] { _ => _.Match(m => m .OnField("_all") .QueryString(Input.Query)), _ => _.Fuzzy(fd => fd .OnField("_all") .MinSimilarity(0.6) .PrefixLength(1) .Value(Input.Query) .Boost(0.1)) }))));
  • 30. HIGHLIGHT RESULT - ELASTICSEARCH { "query": { // see previous example }, "highlight": { "pre_tags": [ "<span class='highlight'>" ], "post_tags": [ "</span>" ], "fields": { "companyName": { "fragment_size": 100, "number_of_fragments": 1 } } } }
  • 31. HIGHLIGHT RESULT - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight(h => h .PreTags("<span class='highlight'>") .PostTags("</span>") .OnFields(new Action<HighlightFieldDescriptor<Customer>>[] { _ => _ .OnField(c => c.CompanyName) .NumberOfFragments(1) .FragmentSize(100) })));
  • 32. FACETS - ELASTICSEARCH { "query": { /* See previous example */ }, "highlight": { /* See previous example */ }, "facets": { "products.productName": { "nested": "products", "terms": { "field": "products.productName", "size": 1000 } }, "products.categoryName": { "nested": "products", "terms": { "field": "products.categoryName", "size": 1000 } }, "country": { "terms": { "field": "country", "size": 1000 } } } }
  • 33. FACETS - NEST _client.Search<Customer>(sd => sd .Query( /* See previous example */ ) .Highlight( /* See previous example */ ) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].ProductName) .Size(1000)) .FacetTerm(f => f .Nested(c => c.Products) .OnField(c => c.Products[0].CategoryName) .Size(1000)) .FacetTerm(f => f .OnField(c => c.Country) .Size(1000)));
  • 35. FILTERS - ELASTICSEARCH { "query": { "filtered": { "query": { /* See previous example */ }, "filter": { "bool": { "must": [ { "terms": { "country": ["usa"] } }, { "nested": { "query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } }, "path": "products" } }, { "nested": { "query": { "terms": { "products.productName": ["Chai"] } }, "path": "products" } } ] } } } }, "facets": { /* See previous example */}, "highlight": { /* See previous example */ } }
  • 36. FILTERS – NEST – PART 1, THE CUSTOMERS FILTER private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); }
  • 37. FILTERS – NEST – PART 1, THE PRODUCTS FILTER private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); }
  • 38. FILTERS – NEST – PART 1, THE MAGIC DICTIONARY public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };
  • 39. FILTERS – NEST – PART 1, ALL THE HELPERS private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Terms(propExpr, items.ToArray()); } private static BaseFilter AddProductsFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr) { return Filter<Customer>.Nested(sel => sel .Path(c => c.Products) .Query(q => q.Terms(propExpr, items.ToArray()))); } public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc = new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>() { {"products.productName", ps => AddProductsFilter(ps, c => c .Products[0].ProductName)}, {"products.categoryName", cs => AddProductsFilter(cs, c => c .Products[0].CategoryName)}, {"country", cs => AddCustomerFilter(cs, c => c.Country)} };
  • 40. FILTERS – NEST – PART 2, THE QUERY _client.Search<Customer>(sd => sd .Query(q => q .Filtered(fq => { fq.Query(qs => { if (!string.IsNullOrEmpty(Input.Query)) { qs.Bool( /* See previous example */ )); } else { qs.MatchAll(); } return qs; }); if (Input.Filter.Count > 0) { var filters = Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray(); fq.Filter(fs => fs.Bool(bf => bf.Must(filters))); } })) .Highlight( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ ) .FacetTerm( /* See previous example */ );
  • 41. SUMMARY Elasticsearch NEST Easy installation Strongly typed client Awesome search engine Fluent Abstraction over Elasticsearch
  • 42. RESOURCES Demo code: https://github.com/mastoj/NestDemo Nest documentation: http://nest.azurewebsites.net/ Nest source code: https://github.com/Mpdreamz/NEST Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense

Editor's Notes

  • #2: Present thetopic Deployment
  • #3: Trenger en ny profilbilde
  • #4: One of the 108 projects that Karel mentioned
  • #5: C:\Dev\git&gt; scriptcsscriptcs (ctrl-c or blank to exit)&gt; usingSystem.Linq.Expressions;&gt; Func&lt;int, int, int&gt; add = (x, y) =&gt; x + y;&gt; add(1, 3)4&gt; Expression&lt;Func&lt;int, int, int&gt;&gt; addExpr = (x, y) =&gt; x + y;&gt; addExpr(1, 3)(1,1): error CS1955: Non-invocablemember &apos;addExpr&apos; cannot be used like a method.&gt; var body = addExpr.Body as BinaryExpression;&gt; Console.WriteLine(body);(x + y)
  • #6: C:\Dev\git&gt; scriptcsscriptcs (ctrl-c or blank to exit)&gt; usingSystem.Linq.Expressions;&gt; Func&lt;int, int, int&gt; add = (x, y) =&gt; x + y;&gt; add(1, 3)4&gt; Expression&lt;Func&lt;int, int, int&gt;&gt; addExpr = (x, y) =&gt; x + y;&gt; addExpr(1, 3)(1,1): error CS1955: Non-invocablemember &apos;addExpr&apos; cannot be used like a method.&gt; var body = addExpr.Body as BinaryExpression;&gt; Console.WriteLine(body);(x + y)
  • #7: C:\Dev\git&gt; scriptcsscriptcs (ctrl-c or blank to exit)&gt; usingSystem.Linq.Expressions;&gt; Func&lt;int, int, int&gt; add = (x, y) =&gt; x + y;&gt; add(1, 3)4&gt; Expression&lt;Func&lt;int, int, int&gt;&gt; addExpr = (x, y) =&gt; x + y;&gt; addExpr(1, 3)(1,1): error CS1955: Non-invocablemember &apos;addExpr&apos; cannot be used like a method.&gt; var body = addExpr.Body as BinaryExpression;&gt; Console.WriteLine(body);(x + y)
  • #8: C:\Dev\git&gt; scriptcsscriptcs (ctrl-c or blank to exit)&gt; usingSystem.Linq.Expressions;&gt; Func&lt;int, int, int&gt; add = (x, y) =&gt; x + y;&gt; add(1, 3)4&gt; Expression&lt;Func&lt;int, int, int&gt;&gt; addExpr = (x, y) =&gt; x + y;&gt; addExpr(1, 3)(1,1): error CS1955: Non-invocablemember &apos;addExpr&apos; cannot be used like a method.&gt; var body = addExpr.Body as BinaryExpression;&gt; Console.WriteLine(body);(x + y)
  • #9: C:\Dev\git&gt; scriptcsscriptcs (ctrl-c or blank to exit)&gt; usingSystem.Linq.Expressions;&gt; Func&lt;int, int, int&gt; add = (x, y) =&gt; x + y;&gt; add(1, 3)4&gt; Expression&lt;Func&lt;int, int, int&gt;&gt; addExpr = (x, y) =&gt; x + y;&gt; addExpr(1, 3)(1,1): error CS1955: Non-invocablemember &apos;addExpr&apos; cannot be used like a method.&gt; var body = addExpr.Body as BinaryExpression;&gt; Console.WriteLine(body);(x + y)