Elasticsearch first-steps
3 likes3,491 views
This document discusses using Elasticsearch for social media analytics and provides examples of common tasks. It introduces Elasticsearch basics like installation, indexing documents, and searching. It also covers more advanced topics like mapping types, facets for aggregations, analyzers, nested and parent/child relations between documents. The document concludes with recommendations on data design, suggesting indexing strategies for different use cases like per user, single index, or partitioning by time range.
1 of 51
Downloaded 37 times
![{
"index":{
"analysis":{
"filter":{
"bigram_shingle_filter":{
"type":"shingle",
"max_shingle_size":2,
"min_shingle_size":2,
...
"analyzer":{
"bigram_analyzer":{
"tokenizer":"whitespace",
"filter":[
"standard",
"bigram_shingle_filter"
]
},
"trigram_analyzer":{
"tokenizer":"whitespace",
"filter":[
"standard",
"trigram_shingle_filter"
]
}
"output_unigrams":"false",
"output_unigrams_if_no_shingles":"fal
se"
},
"trigram_shingle_filter":
{
"type":"shingle",
"max_shingle_size":3,
"min_shingle_size":3,
}
}
"output_unigrams":"false",
"output_unigrams_if_no_shingles":"fal
se"
}
} ...
}
}](https://image.slidesharecdn.com/elasticsearch-first-steps-131203101020-phpapp01/85/Elasticsearch-first-steps-37-320.jpg)
![curl -XGET localhost:9200/authors/nested_author/
_search -d '
{
"query": {
"filtered": {
"query": {"match_all": {}},
"filter": {
"nested": {
"path": "books",
"query":{
"filtered": {
"query": { "match_all": {}},
"filter": {
"and": [
{"term": {"books.publisher":
"penguin"}},
{"term": {"books.genre": "scifi"}}
]](https://image.slidesharecdn.com/elasticsearch-first-steps-131203101020-phpapp01/85/Elasticsearch-first-steps-40-320.jpg)
![Parent and Child query
curl -XPOST localhost:9200/authors/bare_author/
_search -d '{
"query": {
"has_child": {
"type": "book",
"query" : {
"filtered": {
"query": { "match_all": {}},
"filter" : {
"and": [
{"term": {"publisher": "penguin"}},
{"term": {"genre": "scifi"}}
]](https://image.slidesharecdn.com/elasticsearch-first-steps-131203101020-phpapp01/85/Elasticsearch-first-steps-43-320.jpg)
Ad
Recommended
Elasticsearch & "PeopleSearch"
Elasticsearch & "PeopleSearch"George Stathis The document discusses the use of Elasticsearch at Traackr for building a people discovery engine, migrating from Solr in 2012. It covers Elasticsearch's features such as near real-time search, automatic schema detection, and document types, while also addressing search performance challenges and alternatives like Solr grouping and denormalization. The presentation includes examples of search queries, schema mapping, and the advantages of using Elasticsearch over other solutions for real-time search needs.
ElasticSearch - index server used as a document database
ElasticSearch - index server used as a document databaseRobert Lujo The document provides an overview of Elasticsearch, a distributed, schema-free search server based on Apache Lucene, highlighting its features as a document database and its integration in the ELK stack. It emphasizes the use of Elasticsearch for real-time data analysis, multi-tenancy, and full-text search, while also contrasting its functionality with traditional relational databases. Additionally, it discusses hybrid solutions combining Elasticsearch with other databases, use cases, and advantages in document-oriented applications.
Elasticsearch in 15 minutes
Elasticsearch in 15 minutesDavid Pilato This document provides a comprehensive overview of Elasticsearch, starting from its installation and basic usage to advanced features such as query DSL, document indexing, and real-time filtering. It includes practical examples of commands for creating, searching, and analyzing documents, as well as discussing facets and statistical aggregations. The document concludes with suggestions for additional features like bulk operations, percolators, and monitoring APIs.
Elasticsearch for Data Analytics
Elasticsearch for Data AnalyticsFelipe This document provides an overview of using Elasticsearch for data analytics. It discusses various aggregation techniques in Elasticsearch like terms, min/max/avg/sum, cardinality, histogram, date_histogram, and nested aggregations. It also covers mappings, dynamic templates, and general tips for working with aggregations. The main takeaways are that aggregations in Elasticsearch provide insights into data distributions and relationships similarly to GROUP BY in SQL, and that mappings and templates can optimize how data is indexed for aggregation purposes.
Elasticsearch - DevNexus 2015
Elasticsearch - DevNexus 2015Roy Russo This document provides an introduction to Elasticsearch. It begins by introducing the speaker and their background. It then discusses what search is and how search engines work by using an inverted index to map tokens to documents. Elasticsearch is introduced as a search and analytics engine that is document-oriented, distributed, schema-free, and uses HTTP and JSON. It can be used for real-time search and analytics. The document discusses how Elasticsearch is based on Apache Lucene and can be run on multiple nodes in a cluster for high availability. It provides examples of using Elasticsearch for centralized logging, and discusses indexing, querying, and interacting with Elasticsearch via its RESTful API.
ElasticSearch - DevNexus Atlanta - 2014
ElasticSearch - DevNexus Atlanta - 2014Roy Russo This document provides an introduction to Elasticsearch, including:
- Elasticsearch is a document-oriented search engine that uses JSON documents and Apache Lucene for indexing and searching. It allows for horizontal scaling across distributed nodes.
- Elasticsearch is well-suited for full-text search, acting as a fast read database with simple data structures to minimize impedance mismatch. Common uses include logs with Logstash, Elasticsearch, and Kibana.
- Key concepts include nodes, clusters, indexes, mapping types, shards, the inverted index, and the REST API for interacting with Elasticsearch.
elasticsearch - advanced features in practice
elasticsearch - advanced features in practiceJano Suchal The document provides an overview of Elasticsearch and its advanced features, focusing on quick start instructions, data indexing, and search capabilities. It includes a case study on identifying suspicious government contracts using faceted search and heuristics, as well as solutions for percolation, scrolling, and performance optimization. Additionally, it offers links to tutorials and guides for further learning on Elasticsearch.
Elasticsearch
ElasticsearchRicardo Peres Natixis Open Day 2018 presentation about Elasticsearch:
- Elasticsearch is a distributed, RESTful search and analytics engine for indexing and searching JSON documents.
- It allows for distributed logging, document indexing, inexact searches, and custom relevance scoring.
- Documents are organized into indexes, types, and shards for distributed querying and storage.
- Documents can be created, updated, and deleted via REST API calls. Relevance can be customized through boosting, functions, and other scoring methods.
- Kibana provides visualization and analytics capabilities for Elasticsearch data. Logstash and Beats facilitate data collection and shipping.
Elasticsearch Introduction at BigData meetup
Elasticsearch Introduction at BigData meetupEric Rodriguez (Hiring in Lex) The document provides an introduction to Elasticsearch, a distributed search and analytics engine built on Apache Lucene. It outlines its architecture, core features, key components including Logstash and Kibana, and various use cases such as full-text search and data storage. Additionally, it discusses Elasticsearch's capabilities in handling high availability through sharding and replication, along with its extensive ecosystem of integrations and clients.
Introduction to Elasticsearch
Introduction to ElasticsearchJason Austin Elasticsearch is a distributed, RESTful search and analytics engine that allows for fast searching, filtering, and analysis of large volumes of data. It is document-based and stores structured and unstructured data in JSON documents within configurable indices. Documents can be queried using a simple query string syntax or more complex queries using the domain-specific query language. Elasticsearch also supports analytics through aggregations that can perform metrics and bucketing operations on document fields.
Introduction to Elasticsearch with basics of Lucene
Introduction to Elasticsearch with basics of LuceneRahul Jain Rahul Jain gives an introduction to Elasticsearch and its basic concepts like term frequency, inverse document frequency, and boosting. He describes Lucene as a fast, scalable search library that uses inverted indexes. Elasticsearch is introduced as an open source search platform built on Lucene that provides distributed indexing, replication, and load balancing. Logstash and Kibana are also briefly described as tools for collecting, parsing, and visualizing logs in Elasticsearch.
Elastic search Walkthrough
Elastic search WalkthroughSuhel Meman Elasticsearch is an open-source, distributed search and analytics engine built on Apache Lucene. It allows storing, searching, and analyzing large volumes of data quickly and in near real-time. Key concepts include being schema-free, document-oriented, and distributed. Indices can be created to store different types of documents. Mapping defines how documents are indexed. Documents can be added, retrieved, updated, and deleted via RESTful APIs. Queries can be used to search for documents matching search criteria. Faceted search provides aggregated data based on search queries. Elastica provides a PHP client for interacting with Elasticsearch.
Scaling Analytics with elasticsearch
Scaling Analytics with elasticsearchdnoble00 This document summarizes how Elasticsearch can be used for scaling analytics applications. Elasticsearch is an open source, distributed search and analytics engine that can index large volumes of data. It automatically shards and replicates data across nodes for redundancy and high availability. Analytics queries like date histograms, statistical facets, and geospatial searches can retrieve insightful results from large datasets very quickly. The document provides an example of using Elasticsearch to perform sentiment analysis, location tagging, and analytical queries on over 100 million social media documents.
Intro to Elasticsearch
Intro to ElasticsearchClifford James Elasticsearch is an open-source, distributed, real-time document indexer with support for online analytics. It has features like a powerful REST API, schema-less data model, full distribution and high availability, and advanced search capabilities. Documents are indexed into indexes which contain mappings and types. Queries retrieve matching documents from indexes. Analysis converts text into searchable terms using tokenizers, filters, and analyzers. Documents are distributed across shards and replicas for scalability and fault tolerance. The REST APIs can be used to index, search, and inspect the cluster.
Using elasticsearch with rails
Using elasticsearch with railsTom Z Zeng Elasticsearch is a powerful open source search and analytics engine. It allows for full text search capabilities as well as powerful analytics functions. Elasticsearch can be used as both a search engine and as a NoSQL data store. It is easy to set up, use, scale, and maintain. The document provides examples of using Elasticsearch with Rails applications and discusses advanced features such as fuzzy search, autocomplete, and geospatial search.
Solr vs. Elasticsearch - Case by Case
Solr vs. Elasticsearch - Case by CaseAlexandre Rafalovitch The document compares Solr and Elasticsearch, highlighting their similarities and differences, including aspects like full-text search capabilities, configuration methods, and plugin usage. It discusses specific technical details about their setups, indexing processes, and search functionalities, illustrating the strengths and weaknesses of both systems. Additionally, it covers topics such as nested documents, cloud deployment strategies, and explicit versus dynamic mapping in each tool.
Simple search with elastic search
Simple search with elastic searchmarkstory Elasticsearch is a JSON document database that allows for powerful full-text search capabilities. It uses Lucene under the hood for indexing and search. Documents are stored in indexes and types which are analogous to tables in a relational database. Documents can be created, read, updated, and deleted via a RESTful API. Searches can be performed across multiple indexes and types. Elasticsearch offers advanced search features like facets, highlighting, and custom analyzers. Mappings allow for customization of how documents are indexed. Shards and replicas improve performance and availability. Multi-tenancy can be achieved through separate indexes or filters.
Data modeling for Elasticsearch
Data modeling for ElasticsearchFlorian Hopf The document outlines data modeling concepts for Elasticsearch, including storing, querying, and updating various types of data such as strings, numbers, dates, and embedded documents. It discusses the advantages of using a flexible schema and the significance of mapping for efficient querying. The presentation also touches on advanced topics like nested documents, parent-child relationships, and index design strategies for optimal performance.
Intro to elasticsearch
Intro to elasticsearchJoey Wen 1) The document discusses information retrieval and search engines. It describes how search engines work by indexing documents, building inverted indexes, and allowing users to search indexed terms.
2) It then focuses on Elasticsearch, describing it as a distributed, open source search and analytics engine that allows for real-time search, analytics, and storage of schema-free JSON documents.
3) The key concepts of Elasticsearch include clusters, nodes, indexes, types, shards, and documents. Clusters hold the data and provide search capabilities across nodes.
An Introduction to Elastic Search.
An Introduction to Elastic Search.Jurriaan Persyn The document discusses technical details of implementing full-text search using PHP/MySQL and highlights advanced features of Elasticsearch, a distributed search engine built on Apache Lucene. It covers aspects such as data indexing, tokenization, stemming, filtering, and relevance scoring. Additionally, it compares Elasticsearch with other search engines like Solr and Sphinx, and outlines the benefits of using Elasticsearch for efficient and scalable search solutions.
Elasticsearch: You know, for search! and more!
Elasticsearch: You know, for search! and more!Philips Kokoh Prasetyo Elasticsearch is presented as an expert in real-time search, aggregation, and analytics. The document outlines Elasticsearch concepts like indexing, mapping, analysis, and the query DSL. Examples are provided for real-time search queries, aggregations including terms, date histograms, and geo distance. Lessons learned from using Elasticsearch at LARC are also discussed.
Elastic search overview
Elastic search overviewABC Talks The document discusses the concept of big data, highlighting its characteristics like volume, variety, and velocity, as well as the evolution of NoSQL databases designed to handle large and complex datasets. It provides an introduction to Elasticsearch, a distributed open-source search engine built on Apache Lucene, detailing its features, architecture, and operational capabilities such as horizontal scaling and RESTful API usage. Comparisons between RDBMS and NoSQL databases, as well as the CAP theorem, are also explored in relation to data processing needs and architectural considerations.
Introduction to Elasticsearch
Introduction to ElasticsearchSperasoft The document provides an overview of Elasticsearch including that it is easy to install, horizontally scalable, and highly available. It discusses Elasticsearch's core search capabilities using Lucene and how data can be stored and retrieved. The document also covers Elasticsearch's distributed nature, plugins, scripts, custom analyzers, and other features like aggregations, filtering and sorting.
Searching Relational Data with Elasticsearch
Searching Relational Data with Elasticsearchsirensolutions The document discusses various methods for searching relational data using Elasticsearch, emphasizing different approaches like index-time joins and query-time joins. It details the pros and cons of each method, including performance, scalability, and flexibility, while also introducing the Siren plugin to enhance nested data handling. Each solution has unique trade-offs affecting performance and data structure management.
ElasticSearch AJUG 2013
ElasticSearch AJUG 2013Roy Russo This document provides an introduction to Elasticsearch, covering the basics, concepts, data structure, inverted index, REST API, bulk API, percolator, Java integration, and topics not covered. It discusses how Elasticsearch is a document-oriented search engine that allows indexing and searching of JSON documents without schemas. Documents are distributed across shards and replicas for horizontal scaling and high availability. The REST API and query DSL allow full-text search and filtering of documents.
Query DSL In Elasticsearch
Query DSL In ElasticsearchKnoldus Inc. The document provides an overview of Elasticsearch and its Query DSL, describing its functionalities, including types of queries such as full text, term level, and compound queries. It explains the distinction between queries and filters, and details various query types with examples. Additionally, it covers specialized querying techniques, including geo queries and span queries, along with references for further reading.
Elasticsearch Introduction
Elasticsearch IntroductionRoopendra Vishwakarma Elasticsearch is a distributed, open source search and analytics engine built on Apache Lucene. It allows storing and searching of documents of any schema in JSON format. Documents are organized into indexes which can have multiple shards and replicas for scalability and high availability. Elasticsearch provides a RESTful API and can be easily extended with plugins. It is widely used for full-text search, structured search, analytics and more in applications requiring real-time search and analytics of large volumes of data.
Elasticsearch presentation 1
Elasticsearch presentation 1Maruf Hassan This document provides an introduction and overview of Elasticsearch. It discusses installing Elasticsearch and configuring it through the elasticsearch.yml file. It describes tools like Marvel and Sense that can be used for monitoring Elasticsearch. Key terms used in Elasticsearch like nodes, clusters, indices, and documents are explained. The document outlines how to index and retrieve data from Elasticsearch through its RESTful API using either search lite queries or the query DSL.
Elasticsearch - basics and beyond
Elasticsearch - basics and beyondErnesto Reig Elasticsearch is a distributed, scalable, and highly available search and analytics engine. The key points discussed in the document include proper cluster configuration, data mappings, and monitoring. The document outlines Elasticsearch's architecture including shards, replicas, and cluster topology. It also discusses modeling data through mappings, analysis, and handling relationships. Monitoring is important and can be done through Elasticsearch plugins, JVM tools, and the stats API.
Elasticsearch
ElasticsearchAmine Ferchichi This document provides an overview of Elasticsearch, including its uses cases at companies like GitHub, Stack Overflow, and Netflix. It discusses Elasticsearch's data indexing and querying capabilities. Key topics covered include document mapping and types, shards and replicas, analyzers, term queries, match queries, sorting, aggregations, and cluster configuration. The document concludes with lessons learned and a reference to Elasticsearch's documentation.
More Related Content
What's hot (20)
Elasticsearch Introduction at BigData meetup
Elasticsearch Introduction at BigData meetupEric Rodriguez (Hiring in Lex) The document provides an introduction to Elasticsearch, a distributed search and analytics engine built on Apache Lucene. It outlines its architecture, core features, key components including Logstash and Kibana, and various use cases such as full-text search and data storage. Additionally, it discusses Elasticsearch's capabilities in handling high availability through sharding and replication, along with its extensive ecosystem of integrations and clients.
Introduction to Elasticsearch
Introduction to ElasticsearchJason Austin Elasticsearch is a distributed, RESTful search and analytics engine that allows for fast searching, filtering, and analysis of large volumes of data. It is document-based and stores structured and unstructured data in JSON documents within configurable indices. Documents can be queried using a simple query string syntax or more complex queries using the domain-specific query language. Elasticsearch also supports analytics through aggregations that can perform metrics and bucketing operations on document fields.
Introduction to Elasticsearch with basics of Lucene
Introduction to Elasticsearch with basics of LuceneRahul Jain Rahul Jain gives an introduction to Elasticsearch and its basic concepts like term frequency, inverse document frequency, and boosting. He describes Lucene as a fast, scalable search library that uses inverted indexes. Elasticsearch is introduced as an open source search platform built on Lucene that provides distributed indexing, replication, and load balancing. Logstash and Kibana are also briefly described as tools for collecting, parsing, and visualizing logs in Elasticsearch.
Elastic search Walkthrough
Elastic search WalkthroughSuhel Meman Elasticsearch is an open-source, distributed search and analytics engine built on Apache Lucene. It allows storing, searching, and analyzing large volumes of data quickly and in near real-time. Key concepts include being schema-free, document-oriented, and distributed. Indices can be created to store different types of documents. Mapping defines how documents are indexed. Documents can be added, retrieved, updated, and deleted via RESTful APIs. Queries can be used to search for documents matching search criteria. Faceted search provides aggregated data based on search queries. Elastica provides a PHP client for interacting with Elasticsearch.
Scaling Analytics with elasticsearch
Scaling Analytics with elasticsearchdnoble00 This document summarizes how Elasticsearch can be used for scaling analytics applications. Elasticsearch is an open source, distributed search and analytics engine that can index large volumes of data. It automatically shards and replicates data across nodes for redundancy and high availability. Analytics queries like date histograms, statistical facets, and geospatial searches can retrieve insightful results from large datasets very quickly. The document provides an example of using Elasticsearch to perform sentiment analysis, location tagging, and analytical queries on over 100 million social media documents.
Intro to Elasticsearch
Intro to ElasticsearchClifford James Elasticsearch is an open-source, distributed, real-time document indexer with support for online analytics. It has features like a powerful REST API, schema-less data model, full distribution and high availability, and advanced search capabilities. Documents are indexed into indexes which contain mappings and types. Queries retrieve matching documents from indexes. Analysis converts text into searchable terms using tokenizers, filters, and analyzers. Documents are distributed across shards and replicas for scalability and fault tolerance. The REST APIs can be used to index, search, and inspect the cluster.
Using elasticsearch with rails
Using elasticsearch with railsTom Z Zeng Elasticsearch is a powerful open source search and analytics engine. It allows for full text search capabilities as well as powerful analytics functions. Elasticsearch can be used as both a search engine and as a NoSQL data store. It is easy to set up, use, scale, and maintain. The document provides examples of using Elasticsearch with Rails applications and discusses advanced features such as fuzzy search, autocomplete, and geospatial search.
Solr vs. Elasticsearch - Case by Case
Solr vs. Elasticsearch - Case by CaseAlexandre Rafalovitch The document compares Solr and Elasticsearch, highlighting their similarities and differences, including aspects like full-text search capabilities, configuration methods, and plugin usage. It discusses specific technical details about their setups, indexing processes, and search functionalities, illustrating the strengths and weaknesses of both systems. Additionally, it covers topics such as nested documents, cloud deployment strategies, and explicit versus dynamic mapping in each tool.
Simple search with elastic search
Simple search with elastic searchmarkstory Elasticsearch is a JSON document database that allows for powerful full-text search capabilities. It uses Lucene under the hood for indexing and search. Documents are stored in indexes and types which are analogous to tables in a relational database. Documents can be created, read, updated, and deleted via a RESTful API. Searches can be performed across multiple indexes and types. Elasticsearch offers advanced search features like facets, highlighting, and custom analyzers. Mappings allow for customization of how documents are indexed. Shards and replicas improve performance and availability. Multi-tenancy can be achieved through separate indexes or filters.
Data modeling for Elasticsearch
Data modeling for ElasticsearchFlorian Hopf The document outlines data modeling concepts for Elasticsearch, including storing, querying, and updating various types of data such as strings, numbers, dates, and embedded documents. It discusses the advantages of using a flexible schema and the significance of mapping for efficient querying. The presentation also touches on advanced topics like nested documents, parent-child relationships, and index design strategies for optimal performance.
Intro to elasticsearch
Intro to elasticsearchJoey Wen 1) The document discusses information retrieval and search engines. It describes how search engines work by indexing documents, building inverted indexes, and allowing users to search indexed terms.
2) It then focuses on Elasticsearch, describing it as a distributed, open source search and analytics engine that allows for real-time search, analytics, and storage of schema-free JSON documents.
3) The key concepts of Elasticsearch include clusters, nodes, indexes, types, shards, and documents. Clusters hold the data and provide search capabilities across nodes.
An Introduction to Elastic Search.
An Introduction to Elastic Search.Jurriaan Persyn The document discusses technical details of implementing full-text search using PHP/MySQL and highlights advanced features of Elasticsearch, a distributed search engine built on Apache Lucene. It covers aspects such as data indexing, tokenization, stemming, filtering, and relevance scoring. Additionally, it compares Elasticsearch with other search engines like Solr and Sphinx, and outlines the benefits of using Elasticsearch for efficient and scalable search solutions.
Elasticsearch: You know, for search! and more!
Elasticsearch: You know, for search! and more!Philips Kokoh Prasetyo Elasticsearch is presented as an expert in real-time search, aggregation, and analytics. The document outlines Elasticsearch concepts like indexing, mapping, analysis, and the query DSL. Examples are provided for real-time search queries, aggregations including terms, date histograms, and geo distance. Lessons learned from using Elasticsearch at LARC are also discussed.
Elastic search overview
Elastic search overviewABC Talks The document discusses the concept of big data, highlighting its characteristics like volume, variety, and velocity, as well as the evolution of NoSQL databases designed to handle large and complex datasets. It provides an introduction to Elasticsearch, a distributed open-source search engine built on Apache Lucene, detailing its features, architecture, and operational capabilities such as horizontal scaling and RESTful API usage. Comparisons between RDBMS and NoSQL databases, as well as the CAP theorem, are also explored in relation to data processing needs and architectural considerations.
Introduction to Elasticsearch
Introduction to ElasticsearchSperasoft The document provides an overview of Elasticsearch including that it is easy to install, horizontally scalable, and highly available. It discusses Elasticsearch's core search capabilities using Lucene and how data can be stored and retrieved. The document also covers Elasticsearch's distributed nature, plugins, scripts, custom analyzers, and other features like aggregations, filtering and sorting.
Searching Relational Data with Elasticsearch
Searching Relational Data with Elasticsearchsirensolutions The document discusses various methods for searching relational data using Elasticsearch, emphasizing different approaches like index-time joins and query-time joins. It details the pros and cons of each method, including performance, scalability, and flexibility, while also introducing the Siren plugin to enhance nested data handling. Each solution has unique trade-offs affecting performance and data structure management.
ElasticSearch AJUG 2013
ElasticSearch AJUG 2013Roy Russo This document provides an introduction to Elasticsearch, covering the basics, concepts, data structure, inverted index, REST API, bulk API, percolator, Java integration, and topics not covered. It discusses how Elasticsearch is a document-oriented search engine that allows indexing and searching of JSON documents without schemas. Documents are distributed across shards and replicas for horizontal scaling and high availability. The REST API and query DSL allow full-text search and filtering of documents.
Query DSL In Elasticsearch
Query DSL In ElasticsearchKnoldus Inc. The document provides an overview of Elasticsearch and its Query DSL, describing its functionalities, including types of queries such as full text, term level, and compound queries. It explains the distinction between queries and filters, and details various query types with examples. Additionally, it covers specialized querying techniques, including geo queries and span queries, along with references for further reading.
Elasticsearch Introduction
Elasticsearch IntroductionRoopendra Vishwakarma Elasticsearch is a distributed, open source search and analytics engine built on Apache Lucene. It allows storing and searching of documents of any schema in JSON format. Documents are organized into indexes which can have multiple shards and replicas for scalability and high availability. Elasticsearch provides a RESTful API and can be easily extended with plugins. It is widely used for full-text search, structured search, analytics and more in applications requiring real-time search and analytics of large volumes of data.
Elasticsearch presentation 1
Elasticsearch presentation 1Maruf Hassan This document provides an introduction and overview of Elasticsearch. It discusses installing Elasticsearch and configuring it through the elasticsearch.yml file. It describes tools like Marvel and Sense that can be used for monitoring Elasticsearch. Key terms used in Elasticsearch like nodes, clusters, indices, and documents are explained. The document outlines how to index and retrieve data from Elasticsearch through its RESTful API using either search lite queries or the query DSL.
Similar to Elasticsearch first-steps (20)
Elasticsearch - basics and beyond
Elasticsearch - basics and beyondErnesto Reig Elasticsearch is a distributed, scalable, and highly available search and analytics engine. The key points discussed in the document include proper cluster configuration, data mappings, and monitoring. The document outlines Elasticsearch's architecture including shards, replicas, and cluster topology. It also discusses modeling data through mappings, analysis, and handling relationships. Monitoring is important and can be done through Elasticsearch plugins, JVM tools, and the stats API.
Elasticsearch
ElasticsearchAmine Ferchichi This document provides an overview of Elasticsearch, including its uses cases at companies like GitHub, Stack Overflow, and Netflix. It discusses Elasticsearch's data indexing and querying capabilities. Key topics covered include document mapping and types, shards and replicas, analyzers, term queries, match queries, sorting, aggregations, and cluster configuration. The document concludes with lessons learned and a reference to Elasticsearch's documentation.
Elasticsearch as a search alternative to a relational database
Elasticsearch as a search alternative to a relational databaseKristijan Duvnjak The document provides an introduction to Elasticsearch, highlighting its features as a distributed, schema-free search engine built on Apache Lucene, suitable for users transitioning from relational databases. It details the structure of Elasticsearch clusters, indexing, shards, and the importance of JSON-based documents, as well as describing how data retrieval and search functionalities work. Additionally, it outlines practical use cases and performance benefits of Elasticsearch in comparison to traditional databases.
JavaCro'15 - Elasticsearch as a search alternative to a relational database -...
JavaCro'15 - Elasticsearch as a search alternative to a relational database -...HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association Elasticsearch is a distributed, RESTful search and analytics engine built on Apache Lucene. It allows storing and searching of documents in near real-time. Documents are stored in indexes which can be sharded across multiple nodes for horizontal scalability and high availability. Queries use a simple JSON over HTTP interface to retrieve and analyze documents. The PBZ bank uses Elasticsearch to index over 600 million documents from customer transactions for fast retrieval of turnovers by account number.
Elasticsearch
ElasticsearchYervand Aghababyan The document provides an overview of Elasticsearch, comparing it to traditional SQL databases and discussing its advantages in handling complex, flexible searches and indexing. Key features include full-text indexing, relevance scoring, geocoding, and the ability to manage nested documents, with an emphasis on ease of use and scalability. It also addresses common challenges in search functionality and the importance of performance monitoring and optimization.
Elasticsearch and Spark
Elasticsearch and SparkAudible, Inc. Elasticsearch and Spark is a presentation about integrating Elasticsearch and Spark for text searching and analysis. It introduces Elasticsearch and Spark, how they can be used together, and the benefits they provide for full-text searching, indexing, and analyzing large amounts of textual data.
Real time analytics using Hadoop and Elasticsearch
Real time analytics using Hadoop and ElasticsearchAbhishek Andhavarapu This document discusses using Hadoop and Elasticsearch for real-time analytics. It provides an overview of Elasticsearch, including how it is document-oriented, schema-free, distributed and fast. It also demonstrates indexing, retrieving, updating and deleting documents from Elasticsearch. The demo portion involves extracting data from a SQL database using Hive, transforming it with Hadoop/Hive, and loading it into Elasticsearch to run queries. Lessons learned focus on concurrency, filtering, field data caching and JVM memory usage.
Elasticsearch, Logstash, Kibana. Cool search, analytics, data mining and more...
Elasticsearch, Logstash, Kibana. Cool search, analytics, data mining and more...Oleksiy Panchenko The document provides an overview of Elasticsearch, including its architecture, deployment, and ecosystem. It features a demo, real-life case studies, and details on the Elasticsearch query API and relevant concepts like filtering, sharding, and relevance scoring. Additionally, it discusses Elasticsearch integration with various programming languages and tools, illustrating its capabilities in full-text search and data management.
Elasticsearch V/s Relational Database
Elasticsearch V/s Relational DatabaseRicha Budhraja The document compares Elasticsearch, a NoSQL database, to relational databases highlighting their differences in structure, use cases, and relevance-based searching capabilities. It details Elasticsearch's various search functionalities, including full-text analysis, synonym and phonetic searches, as well as its role in log analysis through the ELK stack. Additionally, it explains core Elasticsearch terminology and demonstrates CRUD operations, covering index and document creation, reading, updating, and deleting documents.
Elastic search from the trenches
Elastic search from the trenchesVinícius Carvalho The document discusses the implementation of Elasticsearch at emusic.com to enhance music search relevancy and user experience. Key lessons include understanding sharding and data flow design, managing caching and memory, and the importance of user interaction tracking for search relevance. Overall, the transition to Elasticsearch significantly improved query response times and reduced infrastructure size while enhancing result quality.
Elasticsearch - Scalability and Multitenancy
Elasticsearch - Scalability and MultitenancyBozhidar Bozhanov The document discusses scalability and multitenancy in Elasticsearch, highlighting the challenges of processing large amounts of data and managing multiple users while segregating their data. It covers various indexing structures, recommended shard sizes, and effective search strategies, as well as the complexities of maintaining a production environment. The conclusion emphasizes the need to address scalability and multitenancy as application problems rather than operations tasks.
Elasticsearch for beginners
Elasticsearch for beginnersNeil Baker Elasticsearch is a free and open source distributed search and analytics engine. It allows documents to be indexed and searched quickly and at scale. Elasticsearch is built on Apache Lucene and uses RESTful APIs. Documents are stored in JSON format across distributed shards and replicas for fault tolerance and scalability. Elasticsearch is used by many large companies due to its ability to easily scale with data growth and handle advanced search functions.
Elasticsearch selected topics
Elasticsearch selected topicsCube Solutions Elasticsearch concepts include nodes, clusters, shards, and replicas. Nodes can be master-eligible, data, or client nodes. Shards hold index data and can have replicas for redundancy. The mapping process defines how documents are stored. Analyzers tokenize text for indexing and searching. Thread pools manage threads for different operations. Capacity planning involves calculating needed shards based on data size. Configurations specify settings like shards, replicas, heap size, and disabling swapping.
About elasticsearch
About elasticsearchMinsoo Jun Elasticsearch is an open source search engine based on Lucene. It allows for distributed, highly available, and real-time search and analytics of documents. Documents are indexed and stored across multiple nodes in a cluster, with the ability to scale horizontally by adding more nodes. Elasticsearch uses an inverted index to allow fast full-text searches of documents.
ElasticSearch: Distributed Multitenant NoSQL Datastore and Search Engine
ElasticSearch: Distributed Multitenant NoSQL Datastore and Search EngineDaniel N The document provides an overview of a distributed multitenant NoSQL datastore and search engine, emphasizing concepts such as SQL vs NoSQL, the architecture of Elasticsearch, and core components like indices, shards, and nodes. It highlights the capabilities of Elasticsearch, including cloud compatibility, RESTful API support, and its scalability options. Additionally, it discusses practical usage examples, typical queries, and integrations with AWS services.
Elasticsearch in 15 Minutes
Elasticsearch in 15 MinutesKarel Minarik The document provides an overview of using Elasticsearch. It demonstrates how to install Elasticsearch, index and update documents, perform searches, add nodes to the cluster, and configure shards and clusters. It also covers JSON and HTTP usage, different search types (terms, phrases, wildcards), filtering and boosting searches, and the JSON query DSL. Faceted searching is demonstrated using terms and terms_stats facets. Advanced topics like mapping, analyzing, and features above the basic search capabilities are also briefly mentioned.
Big data elasticsearch practical
Big data elasticsearch practicalJWORKS powered by Ordina This document provides an overview and instructions for setting up Elasticsearch. It discusses:
- How to set up the Elasticsearch workshop environment by installing required software and cloning the GitHub repository.
- Key concepts about Elasticsearch including its distributed and schema-free nature, how it is document oriented, and how indexes, types, documents, and fields relate to a relational database.
- Core components like clusters, nodes, shards, and replicas. It also distinguishes between filters and queries.
- Steps for connecting to Elasticsearch, inserting, searching, updating and deleting data.
- Advanced search techniques including filters, multi-field search, and human language processing using analyzers, stop words, synonyms and normalization
Making your elastic cluster perform - Jettro Coenradie - Codemotion Amsterdam...
Making your elastic cluster perform - Jettro Coenradie - Codemotion Amsterdam...Codemotion The document provides a comprehensive guide on optimizing performance for an Elasticsearch cluster, covering installation, configuration, indexing, and querying strategies. It highlights key considerations such as hardware selection, index design, and mapping while offering practical command examples for configuration and data management. Additionally, it emphasizes monitoring tools and logging settings to enhance cluster visibility and performance troubleshooting.
Elastic pivorak
Elastic pivorakPivorak MeetUp The document provides an overview of Elasticsearch and how it can be used to make software smarter. It discusses how Elasticsearch works and its advantages over other search technologies like SQL and Sphinx. The document also includes case studies of four projects that used Elasticsearch for tasks like search, recommendations, and parsing classified ads. It covers how to install and configure Elasticsearch, as well as how to query an Elasticsearch index through its RESTful API.
Elasticsearch War Stories
Elasticsearch War StoriesArno Broekhof Arno Broekhof, a data engineer at Dataworkz, shares insights on Elasticsearch, its structure, and usage alongside its history since its release in 2010. Key points include the importance of shard management, querying capabilities, and emphasizing the distinction between using queries and filters for efficiency. The document also highlights future developments and best practices for memory and data handling in Elasticsearch.
JavaCro'15 - Elasticsearch as a search alternative to a relational database -...
JavaCro'15 - Elasticsearch as a search alternative to a relational database -...HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
Ad
Recently uploaded (20)
Techniques for Automatic Device Identification and Network Assignment.pdf
Techniques for Automatic Device Identification and Network Assignment.pdfPriyanka Aash Techniques for Automatic Device Identification and Network Assignment
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech EnthusiastRaman Bhaumik A Junior Software Developer with a flair for innovation, Raman Bhaumik excels in delivering scalable web solutions. With three years of experience and a solid foundation in Java, Python, JavaScript, and SQL, she has streamlined task tracking by 20% and improved application stability.
"Database isolation: how we deal with hundreds of direct connections to the d...
"Database isolation: how we deal with hundreds of direct connections to the d...Fwdays What can go wrong if you allow each service to access the database directly? In a startup, this seems like a quick and easy solution, but as the system scales, problems appear that no one could have guessed.
In my talk, I'll share Solidgate's experience in transforming its architecture: from the chaos of direct connections to a service-based data access model. I will talk about the transition stages, bottlenecks, and how isolation affected infrastructure support. I will honestly show what worked and what didn't. In short, we will analyze the controversy of this talk.
From Manual to Auto Searching- FME in the Driver's Seat
From Manual to Auto Searching- FME in the Driver's SeatSafe Software Finding a specific car online can be a time-consuming task, especially when checking multiple dealer websites. A few years ago, I faced this exact problem while searching for a particular vehicle in New Zealand. The local classified platform, Trade Me (similar to eBay), wasn’t yielding any results, so I expanded my search to second-hand dealer sites—only to realise that periodically checking each one was going to be tedious. That’s when I noticed something interesting: many of these websites used the same platform to manage their inventories. Recognising this, I reverse-engineered the platform’s structure and built an FME workspace that automated the search process for me. By integrating API calls and setting up periodic checks, I received real-time email alerts when matching cars were listed. In this presentation, I’ll walk through how I used FME to save hours of manual searching by creating a custom car-finding automation system. While FME can’t buy a car for you—yet—it can certainly help you find the one you’re after!
Coordinated Disclosure for ML - What's Different and What's the Same.pdf
Coordinated Disclosure for ML - What's Different and What's the Same.pdfPriyanka Aash Coordinated Disclosure for ML - What's Different and What's the Same
Salesforce Summer '25 Release Frenchgathering.pptx.pdf
Salesforce Summer '25 Release Frenchgathering.pptx.pdfyosra Saidani Salesforce Summer '25 Release Frenchgathering.pptx.pdf
Wenn alles versagt - IBM Tape schützt, was zählt! Und besonders mit dem neust...
Wenn alles versagt - IBM Tape schützt, was zählt! Und besonders mit dem neust...Josef Weingand IBM LTO10
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdf
Oh, the Possibilities - Balancing Innovation and Risk with Generative AI.pdfPriyanka Aash Oh, the Possibilities - Balancing Innovation and Risk with Generative AI
Quantum AI: Where Impossible Becomes Probable
Quantum AI: Where Impossible Becomes ProbableSaikat Basu Imagine combining the "brains" of Artificial Intelligence (AI) with the "super muscles" of Quantum Computing. That's Quantum AI!
It's a new field that uses the mind-bending rules of quantum physics to make AI even more powerful.
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdf
A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy.pdfPriyanka Aash A Constitutional Quagmire - Ethical Minefields of AI, Cyber, and Privacy
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdf
EIS-Webinar-Engineering-Retail-Infrastructure-06-16-2025.pdfEarley Information Science As AI reshapes expectations in retail and B2B commerce, organizations are recognizing a critical reality: meaningful AI outcomes depend on well-structured, adaptable infrastructure. In this session, Seth Earley is joined by Phil Ryan - AI strategist, search technologist, and founder of Glass Leopard Technologies - for a candid conversation on what it truly means to engineer systems for scale, agility, and intelligence.
Phil draws on more than two decades of experience leading search and AI initiatives for enterprise organizations. Together, he and Seth explore the challenges businesses face when legacy architectures limit personalization, agility, and real-time decisioning - and what needs to change to support agentic technologies and next-best-action capabilities.
Key themes from the webinar include:
Composability as a prerequisite for AI - Why modular, loosely coupled systems are essential for adapting to rapid innovation and evolving business needs
Search and relevance as foundational to AI - How techniques honed-in enterprise search have laid the groundwork for more responsive and intelligent customer experiences
From MDM and CDP to agentic systems - How data platforms are evolving to support richer customer context and dynamic orchestration
Engineering for business alignment - Why successful AI programs require architectural decisions grounded in measurable outcomes
The conversation is practical and forward-looking, connecting deep technical understanding with real-world business needs. Whether you’re modernizing your commerce stack or exploring how AI can enhance product discovery, personalization, or customer journeys, this session provides a clear-eyed view of the capabilities, constraints, and priorities that matter most.
AI vs Human Writing: Can You Tell the Difference?
AI vs Human Writing: Can You Tell the Difference?Shashi Sathyanarayana, Ph.D This slide illustrates a side-by-side comparison between human-written, AI-written, and ambiguous content. It highlights subtle cues that help readers assess authenticity, raising essential questions about the future of communication, trust, and thought leadership in the age of generative AI.
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...
" How to survive with 1 billion vectors and not sell a kidney: our low-cost c...Fwdays Let's talk about our history. How we started the project with a small vector database of less than 2 million records. Later, we received a request for +100 million records, then another +100... And so gradually we reached almost 1 billion. Standard tools were quickly running out of steam - we were running into performance, index size, and very limited resources. After a long series of trials and errors, we built our own low-cost cluster, which today stably processes thousands of queries to more than 1B vectors.
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdf
Cracking the Code - Unveiling Synergies Between Open Source Security and AI.pdfPriyanka Aash Cracking the Code - Unveiling Synergies Between Open Source Security and AI
OpenPOWER Foundation & Open-Source Core Innovations
OpenPOWER Foundation & Open-Source Core InnovationsIBM penPOWER offers a fully open, royalty-free CPU architecture for custom chip design.
It enables both lightweight FPGA cores (like Microwatt) and high-performance processors (like POWER10).
Developers have full access to source code, specs, and tools for end-to-end chip creation.
It supports AI, HPC, cloud, and embedded workloads with proven performance.
Backed by a global community, it fosters innovation, education, and collaboration.
Connecting Data and Intelligence: The Role of FME in Machine Learning
Connecting Data and Intelligence: The Role of FME in Machine LearningSafe Software In this presentation, we want to explore powerful data integration and preparation for Machine Learning. FME is known for its ability to manipulate and transform geospatial data, connecting diverse data sources into efficient and automated workflows. By integrating FME with Machine Learning techniques, it is possible to transform raw data into valuable insights faster and more accurately, enabling intelligent analysis and data-driven decision making.
2025_06_18 - OpenMetadata Community Meeting.pdf
2025_06_18 - OpenMetadata Community Meeting.pdfOpenMetadata The community meetup was held Wednesday June 18, 2025 @ 9:00 AM PST.
Catch the next OpenMetadata Community Meetup @ https://www.meetup.com/openmetadata-meetup-group/
In this month's OpenMetadata Community Meetup, "Enforcing Quality & SLAs with OpenMetadata Data Contracts," we covered data contracts, why they matter, and how to implement them in OpenMetadata to increase the quality of your data assets!
Agenda Highlights:
👋 Introducing Data Contracts: An agreement between data producers and consumers
📝 Data Contracts key components: Understanding a contract and its purpose
🧑🎨 Writing your first contract: How to create your own contracts in OpenMetadata
🦾 An OpenMetadata MCP Server update!
➕ And More!
cnc-processing-centers-centateq-p-110-en.pdf
cnc-processing-centers-centateq-p-110-en.pdfAmirStern2 מרכז עיבודים תעשייתי בעל 3/4/5 צירים, עד 22 החלפות כלים עם כל אפשרויות העיבוד הדרושות. בעל שטח עבודה גדול ומחשב נוח וקל להפעלה בשפה העברית/רוסית/אנגלית/ספרדית/ערבית ועוד..
מסוגל לבצע פעולות עיבוד שונות המתאימות לענפים שונים: קידוח אנכי, אופקי, ניסור, וכרסום אנכי.
"Scaling in space and time with Temporal", Andriy Lupa.pdf
"Scaling in space and time with Temporal", Andriy Lupa.pdfFwdays Design patterns like Event Sourcing and Event Streaming have long become standards for building real-time analytics systems. However, when the system load becomes nonlinear with fast and often unpredictable spikes, it's crucial to respond quickly in order not to lose real-time operating itself.
In this talk, I’ll share my experience implementing and using a tool like Temporal.io. We'll explore the evolution of our system for maintaining real-time report generation and discuss how we use Temporal both for short-lived pipelines and long-running background tasks.
Ad
Elasticsearch first-steps
- 1. Elasticsearch: first steps with an Aggregate-oriented database Jug Roma 28/11/2013 Matteo Moci
- 2. Me Matteo Moci @matteomoci http://mox.fm Software Engineer R&D, new product development
- 3. Agenda • 2 Use cases • Elasticsearch Basics • Data Design for scaling
- 4. Social Media Analytics Platform for Marketing Agencies
- 5. Scenario • Using Elasticsearch as: • Analytics engine Aggregate repository •
- 6. Use case 1 • count values distribution over time
- 7. Before • ~10M documents • Heaviest query: ~10 minutes • • Our staff had a problem
- 8. After • ~10M documents • Heaviest query: ~1 second (also with larger • dataset)
- 9. Use case 2 • Aggregate-oriented repository • ...as in DDD http://ptgmedia.pearsoncmg.com/images/chap10_9780321834577/elementLinks/10fig05.jpg
- 10. Elasticsearch Distributed RESTful search and analytics real time data and analytics distributed high availability multi tenancy full-text search schema free RESTful, JSON API
- 11. Elasticsearch basics • Install • API • Types mapping • Facets • Relations
- 12. Install $ wget https:// download.elasticsearch.org/... $ tar -xf elasticsearch-0.90.7.tar.gz
- 13. Run!
- 15. Run! $ ./elasticsearch-0.90.7/bin/ elasticsearch -f $ ./elasticsearch-0.90.7/bin/ elasticsearch -f es Hulk
- 16. Run! $ ./elasticsearch-0.90.7/bin/ elasticsearch -f $ ./elasticsearch-0.90.7/bin/ elasticsearch -f es Hulk Thor
- 17. Index a document $ curl -X PUT localhost:9200/ products/product/1 -d '{ "name" : "Camera" }'
- 26. Async Java API this.client.prepareGet("documents", "document", id) //async, non blocking APIs //use a listener to handle result. non-blocking .execute(new ActionListener<GetResponse>() { @Override public void onResponse(GetResponse getFields) { // } @Override public void onFailure(Throwable e) { // }
- 27. Mapping Mappings define how primitive types are stored and analyzed
- 28. Mapping • JSON data is parsed on indexing • Mapping is done on first field indexing • Inferred if not configured (!) • Types: float, long, boolean, date (+formatting), object, nested • String type can have arbitrary analyzers • Fields can be split up in more fields
- 29. "text": { "type": "multi_field", "fields": { "text": { "type": "string", "index": "analyzed", "index_analyzer": "whitespace", "analyzer": "whitespace" }, "text_bigram": { "type": "string", "index": "analyzed", "index_analyzer": "bigram_analyzer", "search_analyzer": "bigram_analyzer" }, "text_trigram": { "type": "string", "index": "analyzed", "index_analyzer": "trigram_analyzer", "search_analyzer": "trigram_analyzer"
- 30. Mapping - lessons • schema can evolve (e.g. add fields) • inferred if not specified (!) • worst case: reindex • use aliases to enable zero downtime
- 31. Search with Facets final TermsFacetBuilder userFacet = FacetBuilders.termsFacet(MENTION_FACET_NAME) .field(USER_ID).size(maxUsersAmount); SearchResponse response; response = client.prepareSearch(Indices.USERS) .setTypes(USER_TYPE) .setQuery(someQuery).setSize(0) .setSearchType(SearchType.COUNT) .addFacet(userFacet).execute().actionGet() ; final TermsFacet facets = (TermsFacet) response.getFacets().facetsAsMap() .get(MENTION_FACET_NAME);
- 32. Query Facets
- 33. Date Histogram Facet The histogram facet works with numeric data by building a histogram across intervals of the field values. Each value is placed in a “bucket”
- 34. { } "query" : { "match_all" : {} }, "facets" : { "histo1" : { "histogram" : { "field" : "followers", "interval" : 10 } } }
- 35. Facets - lessons • • • Bug in 0.90.x: https://github.com/elasticsearch/elasticsearch/ issues/1305* Solutions: use 1 shard ask for top 100 instead of 10 *will be solved in 1.0 with aggregation module
- 36. Analyzers A Lucene analyzer consists of a tokenizer and an arbitrary amount of filters (+ char filters)
- 38. Relations between Documents Author 1 N Book • nested: faster reads, update needs reindex, cross object match • parent/child: same shard, no reindex on update, difficult sorting
- 39. Nested Documents Specify Book type is “nested” in Author’s Mapping We can query Authors with a query on properties of nested Books “Authors who published at least a book with Penguin, in scifi genre”
- 40. curl -XGET localhost:9200/authors/nested_author/ _search -d ' { "query": { "filtered": { "query": {"match_all": {}}, "filter": { "nested": { "path": "books", "query":{ "filtered": { "query": { "match_all": {}}, "filter": { "and": [ {"term": {"books.publisher": "penguin"}}, {"term": {"books.genre": "scifi"}} ]
- 41. Parent and Child Indexing happens separately Specify _parent type in Child mapping (Book) When indexing Books, specify id of Author
- 42. curl -XPOST localhost:9200/authors/book/_mapping -d '{ "book":{ "_parent": {"type": "bare_author"} } }' curl -XPOST localhost:9200/authors/book/1?parent=2 -d '{ "name": "Revelation Space", "genre": "scifi", "publisher": "penguin" }'
- 43. Parent and Child query curl -XPOST localhost:9200/authors/bare_author/ _search -d '{ "query": { "has_child": { "type": "book", "query" : { "filtered": { "query": { "match_all": {}}, "filter" : { "and": [ {"term": {"publisher": "penguin"}}, {"term": {"genre": "scifi"}} ]
- 44. Data Design Index Configurations • One index “per user” • Single index • SI + Routing: 1 index + custom doc routing • to shards Time: 1 index per time window * * we can search across indices
- 45. One Index per user Hulk Thor User1 s0 User1 s1 User2 s0 + different sharding per user - small users own (and cost) at least 1 shard
- 46. Single Index Hulk Thor Users s0 Users s3 Users s2 + filter by user id, support growth - search hits all shards
- 47. Single Index + routing Hulk Thor Users s0 Users s3 Users s2 + a user’s data is all in one shard, allows large overallocation
- 48. Index per time range Hulk Thor 2013_01 s1 2013_01 s2 2013_02 s1 + allows change in future indices
- 49. Data Design - lessons Test, test, test your use case! Take a single node with one shard and throw load at it, checking the shard capacity The shard is the scaling unit: overallocate to enable future scaling #shards > #nodes
- 50. ...ES has lots of other features! • Bulk operations • Percolator (alerts, classification, …) • Suggesters (“Did you mean …?”) • Index templates (Automatic index • • • configuration) Monitoring API (Amount of memory used, number of operations, …) Plugins ...