index 크기 확인

Splunk의 index 크기를 확인하고 Grafana 대시보드를 생성하는 방법을 알려드리겠습니다.

1. Splunk에서 index 크기 확인 명령어

먼저 Splunk에서 index 크기를 확인하는 기본 SPL 쿼리:

spl
| eventcount summarize=false index=* 
| eval size_GB = round(count / 1024 / 1024 / 1024, 2)
| stats sum(size_GB) as total_size_GB by index
| sort - total_size_GB


또는:

spl
| dbinspect index=* 
| stats sum(sizeOnDiskMB) as size_mb by index
| eval size_gb = round(size_mb / 1024, 2)
| fields index, size_gb
| sort - size_gb


2. Grafana 대시보드 JSON 설정

json
{
  "dashboard": {
    "id": null,
    "title": "Splunk Index Size Monitoring",
    "tags": ["splunk", "index", "storage"],
    "timezone": "browser",
    "panels": [
      {
        "id": 1,
        "title": "Index Size Distribution (GB)",
        "type": "bargauge",
        "targets": [
          {
            "refId": "A",
            "queryType": "custom",
            "rawQuery": true,
            "query": "| dbinspect index=* | stats sum(sizeOnDiskMB) as size_mb by index | eval size_gb = round(size_mb / 1024, 2) | fields index, size_gb"
          }
        ],
        "options": {
          "orientation": "horizontal",
          "displayMode": "gradient",
          "showUnfilled": true
        },
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "continuous-BlPu"
            },
            "thresholds": {
              "mode": "absolute",
              "steps": [
                {"value": null, "color": "green"},
                {"value": 100, "color": "yellow"},
                {"value": 500, "color": "red"}
              ]
            }
          }
        }
      },
      {
        "id": 2,
        "title": "Total Index Size Over Time",
        "type": "timeseries",
        "targets": [
          {
            "refId": "A",
            "queryType": "custom",
            "rawQuery": true,
            "query": "| dbinspect index=* | stats sum(sizeOnDiskMB) as total_size_mb | eval total_size_gb = round(total_size_mb / 1024, 2)"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "color": {
              "mode": "palette-classic"
            },
            "unit": "bytesGB"
          }
        }
      },
      {
        "id": 3,
        "title": "Top 10 Largest Indexes",
        "type": "table",
        "targets": [
          {
            "refId": "A",
            "queryType": "custom",
            "rawQuery": true,
            "query": "| dbinspect index=* | stats sum(sizeOnDiskMB) as size_mb by index | eval size_gb = round(size_mb / 1024, 2) | sort - size_gb | head 10"
          }
        ],
        "options": {
          "showHeader": true,
          "sortBy": [
            {
              "desc": true,
              "displayName": "size_gb"
            }
          ]
        }
      }
    ],
    "time": {
      "from": "now-7d",
      "to": "now"
    },
    "refresh": "5m"
  }
}


3. Grafana 데이터 소스 설정

Splunk 데이터 소스를 Grafana에 추가해야 합니다. `grafana.ini` 또는 데이터 소스 설정:


apiVersion: 1

datasources:
  - name: Splunk
    type: grafana-splunk-datasource
    access: proxy
    url: https://your-splunk-instance:8089
    basicAuth: true
    basicAuthUser: admin
    jsonData:
      username: admin
      password: "your_password"
      defaultIndex: "main"
      tlsSkipVerify: true
    secureJsonData:
      password: "your_encrypted_password"


4. 대시보드 생성 스크립트 (Python 예제)

python
import requests
import json

def create_splunk_dashboard():
    grafana_url = "http://localhost:3000"
    api_key = "your_grafana_api_key"
    
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json"
    }
    
    dashboard_config = {
        # 위에서 정의한 JSON 내용
    }
    
    response = requests.post(
        f"{grafana_url}/api/dashboards/db",
        headers=headers,
        json=dashboard_config
    )
    
    if response.status_code == 200:
        print("Dashboard created successfully!")
    else:
        print(f"Error: {response.status_code}, {response.text}")

create_splunk_dashboard()


5. Docker Compose를 이용한 전체 설정


version: '3'

services:
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_INSTALL_PLUGINS=grafana-splunk-datasource
    volumes:
      - ./grafana/provisioning:/etc/grafana/provisioning


주의사항

1. Splunk 데이터 소스 플러그인이 설치되어 있어야 합니다
2. Splunk 접속 정보와 권한이 properly 설정되어야 합니다
3. 대시보드 JSON에서 쿼리는 실제 Splunk 환경에 맞게 수정해야 합니다
4. 보안을 위해 password는 secure하게 관리해야 합니다

이 설정을 통해 Splunk index 크기를 모니터링하는 Grafana 대시보드를 생성할 수 있습니다.