Installation
- docker run prom/prometheusDocker
- brew install prometheusmacOS
- Download binaryGitHub releases
- Default: :9090Web UI port
Metric Types
- CounterCumulative (only up)
- GaugeUp and down value
- HistogramBucketed values
- SummaryQuantiles
Label Selectors
- =Exact match
- !=Not equal
- =~Regex match
- !~Regex not match
Operators
- + - * / % ^Arithmetic
- == != < > <= >=Comparison
- and or unlessLogical
- on ignoringVector matching
- group_left/rightMany-to-one
PromQL Basic Queries
# Instant vector - current value
http_requests_total
# With label filter
http_requests_total{job="api-server", status="200"}
# Regex match
http_requests_total{job=~"api.*"}
# Range vector - last 5 minutes
http_requests_total[5m]
# Offset - 1 hour ago
http_requests_total offset 1h
Common Functions
# Rate - per-second rate over range
rate(http_requests_total[5m])
# Increase - total increase over range
increase(http_requests_total[1h])
# Histogram percentile
histogram_quantile(0.95, rate(http_duration_bucket[5m]))
# Instant rate (for high-resolution)
irate(http_requests_total[5m])
Aggregation Funcs
- sum()Sum all values
- avg()Average
- min() / max()Min/max value
- count()Count series
- topk() / bottomk()Top/bottom K
- stddev()Std deviation
Time Functions
- rate()Per-second rate
- irate()Instant rate
- increase()Total increase
- delta()Gauge difference
- deriv()Per-second derivative
- predict_linear()Linear prediction
prometheus.yml Config
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: 'prometheus'
static_configs:
- targets: ['localhost:9090']
- job_name: 'node'
static_configs:
- targets: ['localhost:9100']
HTTP API Endpoints
# Query instant vector
GET /api/v1/query?query=up&time=2023-01-01T00:00:00Z
# Query range
GET /api/v1/query_range?query=rate(http_requests_total[5m])&start=...&end=...&step=15s
# List all series
GET /api/v1/series?match[]=up
# Get label values
GET /api/v1/label/job/values
Time Durations
- msMilliseconds
- sSeconds
- mMinutes
- hHours
- dDays
- wWeeks
- yYears
Recording Rules
groups:
- name: example
rules:
- record: job:http_requests:rate5m
expr: sum by (job)(
rate(http_requests_total[5m])
)