Getting started with Elasticsearch and .NET

GETTING STARTED WITH
ELASTICSEARCH ON WINDOWS
AND .NET WITH NEST
A short introduction

Oslo/NNUG Meetup
Tomas Jansson
29/01/2014

THIS IS ME

Tomas Jansson
Manager & Group Lead .NET
BEKK Oslo
@TomasJansson
tomas.jansson@bekk.no
github.com/mastoj
blog.tomasjansson.com

TL;DR;

https://github.com/mastoj/NestDemo

AUDIENCE

N00b

Expert

N00b

Expert

BACKGROUND

This is the data and we need this new application

WHAT I WANT TO SHOW YOU IS...

Elasticsearch is awesome
Indexing using NEST

Querying using NEST
... not about advanced elasticsearch hosting

INSTALLATION

Great news, install as a service added in 0.90.5

Powershell to
the rescue

NEST

Abstraction
over
Elasticsearch

There is an low level abstraction as well called RawElasticClient

NEST

Abstraction
Fluent &
Strongly
over
Elasticsearch
typed

FUNC DEMO

C:Devgit> scriptcs
scriptcs (ctrl-c or blank to exit)

> Func<int, int, int> add = (x, y) => x + y;
> add(1, 3)
4

Func  executable

SIMPLE EXPRESSION DEMO

> using System.Linq.Expressions;
> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;
> addExpr(1, 3)

Expression  ”function description”

(1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method.
> var binExpr = addExpr.Body as BinaryExpression;
> Console.WriteLine(binExpr);
(x + y)
> var add2 = addExpr.Compile();
> add2(3, 1);
4

MORE COMPLEX EXPRESSION DEMO

> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString

MORE COMPLEX EXPRESSION DEMO

> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
Enables us to translate from one domain to another in an ”easy” manner
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString

ELASTICSEARCH CONNECTION

public class ElasticClientWrapper : ElasticClient
{
private static string _connectionString = Settings.ElasticSearchServer;
private static ConnectionSettings _settings =
new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200
.SetDefaultIndex(Settings.Alias) //"customer_product_mapping"
.UsePrettyResponses();
public ElasticClientWrapper()
: base(_settings)
{
}
}
//usage
var client = new ElasticClientWrapper();

MAPPING

public class Product
{
public double UnitPrice { get; set; }
public int TotalQuantity { get; set; }
[ElasticProperty(Index = FieldIndexOption.not_analyzed)]
public string ProductName { get; set; }
[ElasticProperty(Index = FieldIndexOption.not_analyzed)]
public string CategoryName { get; set; }
}
public class Customer
{
public string CustomerID { get; set; }
public string CompanyName { get; set; }
public string Address { get; set; }
public string City { get; set; }
public string Country { get; set; }
[ElasticProperty(Type = FieldType.nested)]
public Product[] Products { get; set; }
}

MAPPING & INDEXING

_client = new ElasticClientWrapper();
Mapping created from attributes
_client.CreateIndex("indexName", s =>
s.AddMapping<Customer>(m => m.MapFromAttributes()));

var customers = _customerRepo.GetCustomers();
_client.IndexMany(customers, "indexName");

Indexing will use the mapping
for the specified index
There is async versions of the
methods

ALIAS

_client.Alias("indexName", "aliasName");

Alias

Index_01

SWAPPING

_client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } );

Alias
1.

Index_02

Swap

3.

Index_01

Create new index

2.

Alias

Delete old index

MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES)

public class SearchModel
{
private int? _numberToTake;
public string Query { get; set; }
public Dictionary<string, IEnumerable<string>> Filter { get; set; }
public int? NumberToTake
{
get { return _numberToTake.HasValue ? _numberToTake.Value : 25; }
set { _numberToTake = value; }
}

}

QUERYING

Elasticsearch

NEST

{

_client.Search<Customer>(sd => sd
.QueryString(Input.Query));

"query": {
"query_string": {
"query": "tomas"
}
}
}

FUZZY

Elasticsearch

NEST

{

.Query(q => q
.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query))));

"query": {
"fuzzy": {
"_all": {
"min_similarity": 0.6,
"prefix_length": 1,
"value": "tomas"
}
}
}
}

Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»

FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH

{
"query": {
"bool": {
"should": [{
"match": {
"_all": {
"query": "tomas"
}
}
},
{
"fuzzy": {
"_all": {
"boost": 0.1,
"min_similarity": 0.6,
"prefix_length": 1,
"value": "tomas"
}
}
}]
}
}
}

FUZZY IMPROVED (USING BOOL QUERY) - NEST

.Query(q => q
.Bool(b => b
.Should(new Func<QueryDescriptor<Customer>, BaseQuery>[]
{
_ => _.Match(m => m
.OnField("_all")
.QueryString(Input.Query)),
_ => _.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query)
.Boost(0.1))
}))));

HIGHLIGHT RESULT - ELASTICSEARCH

{
"query": {
// see previous example
},
"highlight": {
"pre_tags": [
"<span class='highlight'>"
],
"post_tags": [
"</span>"
],
"fields": {
"companyName": {
"fragment_size": 100,
"number_of_fragments": 1
}
}
}

}

HIGHLIGHT RESULT - NEST

.Query( /* See previous example */ )
.Highlight(h => h
.PreTags("<span class='highlight'>")
.PostTags("</span>")
.OnFields(new Action<HighlightFieldDescriptor<Customer>>[]
{
_ => _
.OnField(c => c.CompanyName)
.NumberOfFragments(1)
.FragmentSize(100)
})));

FACETS - ELASTICSEARCH

{
"query": { /* See previous example */ },
"highlight": { /* See previous example */ },
"facets": {
"products.productName": {
"nested": "products",
"terms": { "field": "products.productName", "size": 1000 }
},
"products.categoryName": {
"nested": "products",
"terms": { "field": "products.categoryName", "size": 1000 }
},
"country": {
"terms": { "field": "country", "size": 1000 }
}
}
}

FACETS - NEST

.Query( /* See previous example */ )
.Highlight( /* See previous example */ )
.FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].ProductName)
.Size(1000))
.FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].CategoryName)
.Size(1000))
.FacetTerm(f => f
.OnField(c => c.Country)
.Size(1000)));

http://go-gaga-over-testing.blogspot.no/2011/09/solution-to-warning-in-quality-center.html

FILTERS - ELASTICSEARCH
{
"query": {
"filtered": {
"query": { /* See previous example */ },
"filter": {
"bool": {
"must": [
{
"terms": { "country": ["usa"] }
},
{
"nested": {
"query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } },
"path": "products"
}
},
{
"nested": {
"query": { "terms": { "products.productName": ["Chai"] } },
"path": "products"
}
}
]
}
}
}
},
"facets": { /* See previous example */},
"highlight": { /* See previous example */ }
}

FILTERS – NEST – PART 1, THE CUSTOMERS FILTER

private static BaseFilter AddCustomerFilter(IEnumerable<string> items,
Expression<Func<Customer, object>> propExpr)
{
return Filter<Customer>.Terms(propExpr, items.ToArray());
}

FILTERS – NEST – PART 1, THE PRODUCTS FILTER

private static BaseFilter AddProductsFilter(IEnumerable<string> items,
{
return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));
}

FILTERS – NEST – PART 1, THE MAGIC DICTIONARY

public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =
new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>()
{
{"products.productName", ps => AddProductsFilter(ps, c => c
.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c
.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}
};

FILTERS – NEST – PART 1, ALL THE HELPERS

private static BaseFilter AddCustomerFilter(IEnumerable<string> items,
{
return Filter<Customer>.Terms(propExpr, items.ToArray());
}
private static BaseFilter AddProductsFilter(IEnumerable<string> items,
{
return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));
}
public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =
new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>()
{
{"products.productName", ps => AddProductsFilter(ps, c => c
.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c
.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}
};

FILTERS – NEST – PART 2, THE QUERY
.Query(q => q
.Filtered(fq =>
{
fq.Query(qs =>
{
if (!string.IsNullOrEmpty(Input.Query))
{
qs.Bool( /* See previous example */ ));
}
else
{
qs.MatchAll();
}
return qs;
});
if (Input.Filter.Count > 0)
{
var filters =
Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray();
fq.Filter(fs => fs.Bool(bf => bf.Must(filters)));
}
}))
.Highlight( /* See previous example */ )
.FacetTerm( /* See previous example */ )
.FacetTerm( /* See previous example */ )
.FacetTerm( /* See previous example */ );

SUMMARY

Elasticsearch

NEST

Easy installation

Strongly typed client

Awesome search engine

Fluent
Abstraction over Elasticsearch

RESOURCES

Demo code: https://github.com/mastoj/NestDemo
Nest documentation: http://nest.azurewebsites.net/
Nest source code: https://github.com/Mpdreamz/NEST
Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net
Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense

Getting started with Elasticsearch and .NET

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to Getting started with Elasticsearch and .NET (20)

More from Tomas Jansson (12)

Recently uploaded (20)

Getting started with Elasticsearch and .NET

Editor's Notes