ELK Stack — KQL and Lucene

KQL and Lucene

Kibana supports two query languages for searching data — KQL (Kibana Query Language) and Lucene. Both are used in the Discover query bar, dashboard filters, and visualization editors. Understanding when and how to use each gives you precise control over data filtering.

---

KQL — Kibana Query Language

KQL is Kibana’s default query language. It provides a simplified, user-friendly syntax designed specifically for Kibana. KQL supports autocomplete and is the recommended choice for most use cases.

Field:Value Queries

The most basic KQL query matches a specific field to a value:

log.level: "ERROR"

http.response.status_code: 500

service.name: "payment-service"

For text fields, KQL performs a terms query — it matches if the field contains the value:

message: "connection refused"

This matches documents where the message field contains both “connection” and “refused” (as individual tokens).

Wildcards

KQL supports the * wildcard for partial matching:

host.name: web-server-*

url.path: /api/v2/*

message: *timeout*

Note: Leading wildcards (*timeout) are expensive and slow. Use them sparingly.

Boolean Operators

KQL supports and, or, and not operators (case-insensitive):

log.level: "ERROR" and service.name: "payment-service"

log.level: "ERROR" or log.level: "WARN"

log.level: "ERROR" and not service.name: "test-service"

Grouping with Parentheses

(log.level: "ERROR" or log.level: "WARN") and service.name: "api-gateway"

log.level: "ERROR" and (service.name: "auth-service" or service.name: "payment-service")

Comparison Operators

KQL supports comparison operators for numeric and date fields:

http.response.status_code >= 400

http.response.status_code < 300

event.duration > 5000

@timestamp >= "2026-03-01" and @timestamp < "2026-04-01"

Nested Field Queries

For nested objects, use dot notation:

http.request.method: "POST"

source.geo.country_name: "India"

user.roles: "admin"

Exists Queries

Check if a field exists in a document:

error.message: *

This matches all documents where the error.message field is present and has any value.

---

Lucene Query Syntax

Lucene is the underlying query syntax used by Elasticsearch. It provides more advanced features than KQL, including fuzzy matching, proximity searches, boosting, and regex.

Switching to Lucene in Kibana

1. In the Discover query bar, click the KQL button on the left side.
2. A dropdown appears — select Lucene.
3. The button label changes to Lucene, and you can now write Lucene queries.

Field:Value Queries

log.level:"ERROR"

message:"connection refused"

Note: In Lucene, quotes around the value create a phrase query (exact order match), while without quotes, the terms are matched individually.

Wildcards

host.name:web-server-*

message:time*t

The ? wildcard matches a single character:

host.name:web-server-0?

Regex

Lucene supports regular expressions enclosed in forward slashes:

host.name:/web-server-0[1-5]/

message:/[Ee]rror.*timeout/

Fuzzy Matching (~)

The tilde operator enables fuzzy matching, which finds terms that are similar to the specified term (useful for typos):

message:connecton~

This matches “connection”, “connecton”, “conection”, etc. You can specify the edit distance:

message:connecton~2

Proximity Searches

Find terms within a specified distance of each other:

message:"connection refused"~5

This matches documents where “connection” and “refused” appear within 5 words of each other.

Boosting (^)

Boost the relevance of specific terms:

message:error^3 OR message:warning

This gives documents with “error” in the message three times more relevance weight than documents with “warning”.

Range Queries

Lucene uses brackets for range queries:

http.response.status_code:[400 TO 599]

http.response.status_code:{400 TO 500}

• [ ] — Inclusive (includes the boundary values).
• { } — Exclusive (excludes the boundary values).

Mixed ranges:

http.response.status_code:[400 TO 500}

Date ranges:

@timestamp:["2026-03-01" TO "2026-03-31"]

Boolean Operators

Lucene uses uppercase AND, OR, NOT:

log.level:"ERROR" AND service.name:"payment-service"

log.level:"ERROR" OR log.level:"WARN"

log.level:"ERROR" NOT service.name:"test-service"

You can also use + (must include) and - (must exclude):

+log.level:"ERROR" -service.name:"test-service"

---

KQL vs Lucene — Comparison

Feature	KQL	Lucene
Default in Kibana	Yes	No (must switch)
Autocomplete	Yes	No
Syntax style	Simplified	Full query syntax
Wildcards	* only	* and ?
Regex	No	Yes (/pattern/)
Fuzzy matching	No	Yes (term~)
Proximity search	No	Yes ("term1 term2"~N)
Boosting	No	Yes (term^N)
Range queries	>, <, >=, <=	[TO], {TO}
Boolean operators	and, or, not	AND, OR, NOT, +, -
Phrase matching	Automatic for quoted strings	Explicit with quotes
Nested field queries	Dot notation	Dot notation
Exists check	field: *	_exists_:field
Ease of use	Easier	More complex

When to use KQL: Day-to-day searching, simple filters, when you need autocomplete.

When to use Lucene: Fuzzy matching, regex patterns, proximity searches, boosting, complex range queries.

---

Practical Query Examples

Finding HTTP Errors

KQL:

http.response.status_code >= 400 and service.name: "web-frontend"

Lucene:

http.response.status_code:[400 TO 599] AND service.name:"web-frontend"

Finding Slow Requests

KQL:

event.duration > 3000 and http.request.method: "GET"

Lucene:

event.duration:{3000 TO *} AND http.request.method:"GET"

Searching Across Multiple Services

KQL:

(service.name: "auth-service" or service.name: "payment-service") and log.level: "ERROR"

Lucene:

(service.name:"auth-service" OR service.name:"payment-service") AND log.level:"ERROR"

Finding Specific Error Messages

KQL:

message: "out of memory" and log.level: "ERROR"

Lucene (fuzzy — tolerates typos):

message:memmory~2 AND log.level:"ERROR"

Excluding Test Environments

KQL:

log.level: "ERROR" and not environment: "staging" and not environment: "test"

Lucene:

log.level:"ERROR" -environment:"staging" -environment:"test"

Finding Documents With a Specific Field

KQL:

error.stack_trace: *

Lucene:

_exists_:error.stack_trace

---

Saved Queries

You can save frequently used queries for reuse across Discover, dashboards, and visualizations.

Saving a Query

1. Type your query in the query bar.
2. Click the bookmark icon (💾) on the right side of the query bar.
3. Click Save current query.
4. Enter a name — for example, Production Errors Only.
5. Optionally, check Include filters to save the active filter pills along with the query.
6. Optionally, check Include time range to save the current time picker setting.
7. Click Save.

Loading a Saved Query

1. Click the bookmark icon in the query bar.
2. Browse or search your saved queries.
3. Click a saved query to apply it.

Saved queries are shared across Discover, visualizations, and dashboards — saving you from rewriting the same filters repeatedly.

---

Quick Reference

Task	KQL Example	Lucene Example
Exact match	log.level: "ERROR"	log.level:"ERROR"
Wildcard	host.name: web-*	host.name:web-*
Boolean AND	field1: "a" and field2: "b"	field1:"a" AND field2:"b"
Boolean OR	field1: "a" or field1: "b"	field1:"a" OR field1:"b"
Boolean NOT	not field: "value"	NOT field:"value" or -field:"value"
Greater than	status >= 400	status:[400 TO *]
Range	status >= 400 and status < 500	status:[400 TO 500}
Exists	field: *	_exists_:field
Fuzzy	N/A	term~2
Regex	N/A	field:/pattern/
Proximity	N/A	"word1 word2"~5

---

Next: Visualizations — Learn how to create charts, tables, and gauges from your Elasticsearch data.