Elasticsearch Basics
- IndexDatabase equivalent
- DocumentRow equivalent
- FieldColumn equivalent
- MappingSchema definition
- Default: :9200HTTP API port
Logstash Pipeline
- input { }Data sources
- filter { }Processing
- output { }Destinations
- Default: :5044Beats input port
Kibana Features
- DiscoverLog exploration
- DashboardVisualizations
- Dev ToolsQuery console
- AlertingAlert rules
- Default: :5601Web UI port
Beats Family
- FilebeatLog files
- MetricbeatSystem metrics
- PacketbeatNetwork data
- HeartbeatUptime monitoring
- AuditbeatAudit data
Elasticsearch API - Index Operations
# Create index
PUT /my-index
# Index a document
POST /my-index/_doc
{ "message": "Hello World", "@timestamp": "2024-01-15T10:00:00Z" }
# Get document
GET /my-index/_doc/1
# Delete index
DELETE /my-index
Search Queries
# Match query (full-text search)
GET /logs/_search
{
"query": {
"match": { "message": "error" }
}
}
# Bool query (AND/OR)
{
"query": {
"bool": {
"must": [{ "match": { "status": "error" }}],
"filter": [{ "range": { "@timestamp": { "gte": "now-1h" }}}]
}
}
}
Query Types
- matchFull-text search
- termExact match
- rangeRange filter
- boolCombine queries
- wildcardPattern match
- existsField exists
Aggregations
- termsGroup by field
- date_histogramTime buckets
- avg/sum/min/maxMetrics
- cardinalityUnique count
- percentilesPercentile stats
Logstash Pipeline Example
input {
beats { port => 5044 }
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
date {
match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "logs-%{+YYYY.MM.dd}"
}
}
KQL (Kibana Query)
# Field equals value
status: "error"
# AND / OR
status: error and host: server1
# Wildcard
message: *timeout*
# Range
response_time >= 500
Cluster Health
# Cluster health
GET /_cluster/health
# Node stats
GET /_nodes/stats
# Index stats
GET /my-index/_stats
# Cat APIs
GET /_cat/indices?v
GET /_cat/shards?v