Complete Guide to Elasticsearch

Introduction to the course and to what you will learn.

What is Elasticsearch all about? How does it work, why is it so popular, and who uses it? There are many questions to be asked, and many answers to be given. Get the answers in this lecture.

This lecture walks through the components of the Elastic Stack, giving you an overview of how these fit together with Elasticsearch.

Test your understanding of what the purpose of each part of the Elastic Stack is.

In this lecture, we take a look at some common use cases for the Elastic Stack, and how a simple architecture might evolve over time to incorporate more components of the Elastic Stack.

What the Q&A is for, and what it isn't for.

When it comes to installing Elasticsearch and Kibana, you have a number of options. Check out this quick overview to help decide which approach is best for you.

Running Elasticsearch and Kibana in Elastic Cloud is the easiest way to get up and running quickly.

Learn how to install both Elasticsearch and Kibana on macOS and Linux.

Understand the basic architecture of Elasticsearch, including what nodes, indices, and documents are.

Let's take a moment to see what the cluster that we started up looks like internally.

Using Kibana's Console tool is not the only way you can send requests to Elasticsearch's REST API. Learn how to do the same with the cURL HTTP client in this lecture.

Learn what sharding is, and how it helps scale an Elasticsearch cluster, both in terms of data volume and query throughput.

Let's be sure that you understand the most important concepts of sharding.

Learn what replication is, and how it helps ensure high availability as well as increased query throughput.

Let's be sure that you understand the most important concepts of replication.

Learn how to add more nodes to our single-node Elasticsearch cluster, forming a cluster consisting of three nodes.

Elasticsearch nodes have roles, which can be configured for each node. Get an overview of which roles are available, and when you should use them.

Let's take a quick moment to wrap up this section of the course.

Before adding some data into our cluster, let's see how to both create and delete indices. When creating an index, we will supply two settings for specifying the number of shards and replica shards for the index.

In this lecture, you will learn how to index documents in two ways; by having Elasticsearch generate an ID for documents automatically, and by specifying an ID ourselves.

Learn how to retrieve a document based on its ID.

Learn how to update an existing field's value, and how to add a new field to a document.

Learn how to update documents with scripts. Scripts enable us to update documents dynamically, such as by performing calculations on field values. You will also see how to include parameters within scripts. Lastly, you will learn how to ignore a given document during an update, how to conditionally update a document, and how to delete a document within a script.

In this lecture, we will cover so-called upserts. An upsert is a way to index a document or update an existing one, depending on whether or not a given document already exists.

Apart from updating documents, we can also replace them altogether. You will learn how to do just that in this lecture.

The last document action you need to learn, is to delete documents. Fortunately, this is extremely easy due to Elasticsearch's REST API.

How did Elasticsearch find the documents that we have indexed in the previous lectures? And how did it know where to store the documents that we indexed? That's what we will be looking at in this lecture.

Now that you know what routing is, let's take a closer look at how Elasticsearch reads data. This involves a coordinating node and Adaptive Replica Selection (ARS).

Now that you know how Elasticsearch reads data, let's talk about how it writes data. That's a more complicated subject involving something called primary terms, sequence numbers, as well as global and local checkpoints. We will also cover the basics of how data is replicated to replica shards.

Elasticsearch uses a primitive kind of document versioning. Let's take a look at what that is all about.

Documents might be overwritten inadvertently due to a web server running multiple threads in parallel, for instance. How do we deal with this problem? We first look at how we used to handle this, before diving into how primary terms and sequence numbers can be used to overcome this challenge.

Let's see how we can update multiple documents with a single query, similar to an UPDATE WHERE query for relational databases. We will also go through the inner workings of the query, including how it handles errors and performs optimistic concurrency control.

Now that you know how to update multiple documents with one query, let's do the same thing, just in regards to deleting documents.

Learn how to process multiple (potentially thousands or millions) actions within one query. This can be used to index, update, or delete many documents at the same time.

It's time to import some test data. Learn how to import data by using the Bulk API together with cURL on the command line.

Quick introduction to what is covered in this section of the course.

Introduction to the concept of text analysis.

Learn how to use the Analyze API to test built-in and custom analyzers. It can also be used to test character filters, tokenizers, and token filters.

Learn how text values are analyzed and stored within a data structure called an inverted index.

Introduction to the concept of mapping.

An overview of the most important data types in Elasticsearch, with a focus on the "object," "nested," and "keyword" data types.

Let's take a closer look at how the "keyword" data type works and learn why it is used for exact matches, aggregations, and sorting.

Learn what type coercion is all about and how Elasticsearch can sometimes convert from one data type to another.

How can we index array values into Elasticsearch? How do arrays work, and what are their limitations? Learn the answers to those questions in this lecture.

Time to add our first mapping to an index.

Learn how to retrieve mappings for an index, and also specific field mappings.

Mapping queries can be simplified when using a dot notation syntax for nested fields. Learn how to do that in this lecture.

Sometimes you might want to add field mappings to an existing index. Learn how to do that in this lecture.

Let's take a look at how Elasticsearch handles dates. In particular, which date formats are supported, how timezones work, how Elasticsearch stores dates internally, and how dates are searched by search queries.

What happens if we leave out a field when indexing a document? And does adding a field mapping make a field required? Those are some of the questions that we will answer in this lecture.

In this lecture, we take a look at the most important mapping parameters, including "doc_values," "norms," "index," and more.

In this lecture you will learn how to update existing field mappings. There are quite a few limitations to updating mappings in Elasticsearch, so we will cover those as well.

Let's see how we can reindex documents into a new index. We will take a look at how the Reindex API works and cover a number of common use cases with examples.

Instead of renaming fields, we can make use of field aliases. You will see how to add and use field aliases in this lecture.

An Elasticsearch field can actually be mapped in multiple ways. This is very useful for querying field values in different ways.

Index templates can be used to specify field mappings and index settings. These can then automatically be applied to new indices based on index patterns. We will take a look at how index templates are commonly used.

Time to get into dynamic mapping now. We will take a look at the rules that Elasticsearch uses when dynamically mapping fields, along with a couple of examples.

Now that you know of both explicit and dynamic mapping, what about combining them? As you will see, you don't necessarily have to choose either explicit or dynamic mapping.

Let's take a look at how we can configure dynamic mapping. You will see how to disable it, how to set it to "strict," and how inheritance is supported. Apart from that, we will also look at numeric detection and date detection.

Dynamic mapping can be configured with dynamic templates. These templates define certain conditions, and when they are matched, the specified mapping will be applied.

Before moving on to analysis and analyzers, I have a few recommendations in regards to mapping.

Stemming and stop words are two common things to handle during text analysis. We will look at how words can be stemmed to their root form as a way to improve search results. We will also cover the concept of removing stop words.

If terms are changed during the analysis process (e.g. stemmed or stop words being removed), how do searches work? Find the answer in this lecture.

Let's go through the most important built-in analyzers and also cover how to configure and use them.

In this lecture, we are going to build a custom analyzer. As part of that, I will show you how to configure a built-in token filter as well.

Apart from specifying analyzers when creating a new index, sometimes we might need to add an analyzer to an existing index. Let's see how to do that in this lecture. In the process, we will look at opening and closing indices, as well as static and dynamic index settings.

Let's take a look at how to update analyzers. While doing so is easy, there is a very important thing to be aware of; failing to handle the situation correctly, can potentially lead to big problems.

A quick wrap up of this section.

Let's take a first look at how we can search for data in Elasticsearch. There are two ways; URI searches and using the Query DSL. We will use the Query DSL for searches in this course.

Let's take a look at a group of queries called term level queries. We will look at what characterizes term level queries, as well as a few do's and don'ts.

Let's see how to use term level queries in Elasticsearch, beginning with the "term" query.

If you want to retrieve a number of documents by their IDs, this can be done with the "ids" query. This lecture shows you how it works.

The range query is used to perform range searches in Elasticsearch. It can be used with both numbers and dates, both of which you will see in this lecture.

Term level queries are generally used for exact matching, but we can add a bit of flexibility by searching for prefixes, or by using wildcards or regular expressions.

Let's see how to find documents that have (or don't have) a value for a given field. There are a few gotchas - such as when a field "exists" - which we will take a look at.

Let's test your understanding of term level queries with a little quiz.

Let's now talk about another group of queries, namely full text queries. They are fundamentally different than term level queries. This lecture covers the difference.

The "match" query is probably the most important full text query in Elasticsearch. It's both simple and powerful at the same time. Let's take a look.

We haven't talked about relevance scoring yet. Here is a quick introduction to how relevance scoring works in Elasticsearch.

Sometimes you might want to search multiple fields at the same time, which can be done with the "multi_match" query. There are a few things to be aware of, though, which we will cover in this lecture.

Apart from searching for individual terms, we can also search for phrases. In this case the ordering and positions of terms matter. We will take a look at how this all works under the hood.

In Elasticsearch there are two main categories of queries; leaf queries and compound queries. So what's the difference? Let's take a look.

Time to write some more complex queries. With the "bool" query, we can write both simple and complex boolean logic, while influencing relevance scores of matching documents. Let's take a look at one of the most important queries in Elasticsearch; the "bool" query.

In Elasticsearch, there are two so-called execution contexts; filter and query. Let's take a look at both and see how they differ.

The "boosting" query enables us to reduce relevance scores of documents that match a given criteria (query). This is especially powerful when combined with the "bool" query. This gives us a lot of control over relevance scoring.

Let's take a look at the "dis_max" (disjunction max) query, which is a great way of finding documents that match one or more criteria, while having an influence on relevance scoring. We will also look at how this query relates to the "multi_match" query that we previously looked at.

Remember why we need to use the "nested" data type when we need to query arrays of objects independently? We will take a closer look at that in this lecture and see how to actually query such objects.

So a document has one or more objects that matched a given query. But which objects actually matched the query? Let's see how we can figure this out with Elasticsearch.

Using nested fields is not free, so there are a couple of limitations, which we will take a look at in this lecture.