Prometheus 및 InfluxDB를 장기 스토리지로 사용

Prometheus 및 InfluxDB를 장기 스토리지로 사용

15일보다 오래된 데이터를 저장하기 위해 prometheus를 influxdb에 연결하려고 합니다. promehteus가 influxdb에 연결되어 있지 않으면 Grafana와 Prometheus가 작동합니다.

아래에는 모든 구성 및 매개변수와 함께 오류를 게시했습니다.

Prometheus 1h에서 쿼리 실행은 작동하지만 1w는 작동하지 않습니다. 쿼리 실행 중 오류가 발생했습니다. 다대다 일치는 허용되지 않습니다. 일치하는 레이블은 한쪽에서 고유해야 합니다.

Grafana 쿼리:

node_time{instance=~"$node:$port"} - node_boot_time{instance=~"$node:$port"}

Grafana 오류 응답:

{

  "status": "error",
  "errorType": "execution",
  "error": "many-to-many matching not allowed: matching labels must be unique on one side",
  "message": "many-to-many matching not allowed: matching labels must be unique on one side"
}

Grafana 잘못된 요청

Url   api/datasources/proxy/1/api/v1/query_range?
query=node_time%7Binstance%3D~%22btzhlprom01%3A9100%22%7D%20-%20node_boot_time%7Binstance%3D~%22btzhlprom01%3A9100%22%7D&start=1520764826&end=1520851226&step=1800
Method      GET
X-Grafana-Org-Id 1
Accept      application/json, text/plain, */*

프로메테우스 빌드 정보

Version     2.0.0
Revision    0a74f98628a0463dddc90528220c94de5032d1a0
Branch      HEAD
BuildUser   root@615b82cb36b6
BuildDate   20171108-07:11:59
GoVersion   go1.9.2

Prometheus 명령줄 플래그:

alertmanager.notification-queue-capacity 10000
alertmanager.timeout    10s
completion-bash   false
completion-script-bash false
completion-script-zsh   false
config.file       /opt/prom/prometheus/prometheus.yml
help false
help-long   false
help-man    false
log.level   info
query.lookback-delta    5m
query.max-concurrency   20
query.timeout     2m
storage.tsdb.max-block-duration    36h
storage.tsdb.min-block-duration    2h
storage.tsdb.no-lockfile      false
storage.tsdb.path       /prometheus/
storage.tsdb.retention 15d
version     false
web.console.libraries   /opt/prom/prometheus/console_libraries
web.console.templates   /opt/prom/prometheus/consoles
web.enable-admin-api    false
web.enable-lifecycle    false
web.external-url http://prometheus.apl.dom
web.listen-address      127.0.0.1:9090
web.max-connections     512
web.read-timeout 5m
web.route-prefix /
web.user-assets   

프로메테우스 구성:

global:
  scrape_interval: 10s
  scrape_timeout: 10s
  evaluation_interval: 10s
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - alertmanager.apl.dom:443
    tls_config:
      ca_file: /opt/prom/prometheus/ApprenticeLab.pem
      insecure_skip_verify: true
    scheme: https
    timeout: 10s
rule_files:
- /opt/prom/prometheus/alerts/disk.yml
- /opt/prom/prometheus/alerts/uptime.yml
scrape_configs:
- job_name: prometheus
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: https
  static_configs:
  - targets:
    - prometheus.apl.dom
  tls_config:
    ca_file: /opt/prom/prometheus/TrustedRootCertificate.cer
    insecure_skip_verify: true
- job_name: grafana
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: https
  static_configs:
  - targets:
    - grafana.apl.dom:443
  tls_config:
    ca_file: /opt/prom/prometheus/TrustedRootCertificate.cer
    insecure_skip_verify: true
- job_name: nodes
  scrape_interval: 5s
  scrape_timeout: 5s
  metrics_path: /metrics
  scheme: http
  file_sd_configs:
  - files:
    - /opt/prom/prometheus/targets/infra.json
    - /opt/prom/prometheus/targets/testing.json
    refresh_interval: 5m
remote_write:
- url: http://btzhlinflx01:8086/api/v1/prom/write?u=prometheus&p=prometheus&db=prometheus
  remote_timeout: 30s
  queue_config:
    capacity: 100000
    max_shards: 1000
    max_samples_per_send: 100
    batch_send_deadline: 5s
    max_retries: 10
    min_backoff: 30ms
    max_backoff: 100ms
remote_read:
- url: http://btzhlinflx01:8086/api/v1/prom/read?u=prometheus&p=prometheus&db=prometheus
  remote_timeout: 1m
  read_recent: true

관련 정보