Back to module
Hands-on Lab

#Lab: Prometheus and Grafana Setup

Deploy a complete monitoring stack.

#🎯 Objectives

  • Deploy Prometheus for metrics collection
  • Set up Grafana dashboards
  • Configure alerting rules

#📋 Prerequisites

  • Docker and Docker Compose
  • Basic understanding of metrics

#⏱️ Duration: 45 minutes

#Task 1: Create Docker Compose Stack (10 min)

bash
1mkdir ~/monitoring-lab && cd ~/monitoring-lab
2
3cat << 'EOF' > docker-compose.yml
4version: '3.8'
5
6services:
7  prometheus:
8    image: prom/prometheus:latest
9    ports:
10      - "9090:9090"
11    volumes:
12      - ./prometheus.yml:/etc/prometheus/prometheus.yml
13      - ./alert.rules.yml:/etc/prometheus/alert.rules.yml
14      - prometheus_data:/prometheus
15    command:
16      - '--config.file=/etc/prometheus/prometheus.yml'
17      - '--storage.tsdb.retention.time=15d'
18
19  grafana:
20    image: grafana/grafana:latest
21    ports:
22      - "3000:3000"
23    volumes:
24      - grafana_data:/var/lib/grafana
25    environment:
26      - GF_SECURITY_ADMIN_PASSWORD=admin123
27
28  node-exporter:
29    image: prom/node-exporter:latest
30    ports:
31      - "9100:9100"
32    volumes:
33      - /proc:/host/proc:ro
34      - /sys:/host/sys:ro
35    command:
36      - '--path.procfs=/host/proc'
37      - '--path.sysfs=/host/sys'
38
39volumes:
40  prometheus_data:
41  grafana_data:
42EOF

#Task 2: Configure Prometheus (10 min)

#prometheus.yml

yaml
1global:
2  scrape_interval: 15s
3  evaluation_interval: 15s
4
5alerting:
6  alertmanagers:
7    - static_configs:
8        - targets: []
9
10rule_files:
11  - alert.rules.yml
12
13scrape_configs:
14  - job_name: 'prometheus'
15    static_configs:
16      - targets: ['localhost:9090']
17
18  - job_name: 'node'
19    static_configs:
20      - targets: ['node-exporter:9100']

#alert.rules.yml

yaml
1groups:
2  - name: node-alerts
3    rules:
4      - alert: HighCPU
5        expr: 100 - (avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80
6        for: 5m
7        labels:
8          severity: warning
9        annotations:
10          summary: High CPU usage
11
12      - alert: HighMemory
13        expr: (1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100 > 80
14        for: 5m
15        labels:
16          severity: warning
17        annotations:
18          summary: High memory usage

#Task 3: Start Stack and Verify (5 min)

bash
1docker compose up -d
2
3# Check status
4docker compose ps
5
6# Verify Prometheus
7curl http://localhost:9090/-/healthy
8
9# Check targets
10curl http://localhost:9090/api/v1/targets

#Task 4: Configure Grafana (15 min)

  1. Open http://localhost:3000
  2. Login: admin / admin123
  3. Add Data Source:

#Create Dashboard

Add panels with these queries:

CPU Usage:

promql
100 - (avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

Memory Usage:

promql
(1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100

Disk Usage:

promql
100 - (node_filesystem_avail_bytes / node_filesystem_size_bytes * 100)

#Task 5: Test Queries (5 min)

bash
1# Prometheus API queries
2curl 'http://localhost:9090/api/v1/query?query=up'
3
4curl 'http://localhost:9090/api/v1/query?query=node_memory_MemTotal_bytes'

#✅ Success Criteria

  • Prometheus running on :9090
  • Grafana running on :3000
  • Node exporter metrics visible
  • Dashboard created with panels

#🧹 Cleanup

bash
docker compose down -v
cd ~ && rm -rf monitoring-lab