Prometheus配置监控ip、端口连通，get、post接口连通和状态码

##方法：blackbox_exporter 部署在一台主机上集中配置

第一部分
1.1下载blackbox_exporter安装包
wget https://github.com/prometheus/blackbox_exporter/releases/download/v0.19.0/blackbox_exporter-0.19.0.linux-amd64.tar.gz

tar xf blackbox_exporter-0.19.0.linux-amd64.tar.gz
cd blackbox_exporter-0.19.0.linux-amd64/

1.2，启动配置
cat > /usr/lib/systemd/system/blackbox_exporter.service << EOF

[Unit]
Description=blackbox_exporter
After=network.target

[Service]
User=root
Type=simple
ExecStart=/apps/mon/blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter --config.file=/apps/mon/blackbox_exporter-0.19.0.linux-amd64/blackbox.yml
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

1.3 加载，启动
systemctl daemon-reload
systemctl start blackbox_exporter.service
systemctl enable blackbox_exporter.service
systemctl status blackbox_exporter.service

1.4 blackbox_exporter配置文件
blackbox.yml
位置：/…/blackbox_exporter/blackbox.yml（取决于blackbox_exporter的安装位置）

modules:
http_2xx:
prober: http
http_post_2xx:
prober: http
http:
method: POST
tcp_connect:
prober: tcp
pop3s_banner:
prober: tcp
tcp:
query_response:
- expect: “^+OK”
tls: true
tls_config:
insecure_skip_verify: false
grpc:
prober: grpc
grpc:
tls: true
preferred_ip_protocol: “ip4”
grpc_plain:
prober: grpc
grpc:
tls: false
service: “service1”
ssh_banner:
prober: tcp
tcp:
query_response:
- expect: “^SSH-2.0-”
- send: “SSH-2.0-blackbox-ssh-check”
irc_banner:
prober: tcp
tcp:
query_response:
- send: “NICK prober”
- send: “USER prober prober prober :prober”
- expect: “PING 😦[^ ]+)”
send: “PONG ${1}”
- expect: “^:[ ]+ 001”
icmp:
prober: icmp

上面是自带的，下面是自定义的

http_post_2xx_with_prometheus_post_check_token:
prober: http
http:
method: POST
headers:
Content-Type: application/json #添加头部
body: ‘{“token”:“prometheus_post_check_token”}’ #发送的相关数据

第二部分
prometheus配置
prometheus.yml
位置：/etc/prometheus/prometheus.yml

全局配置

global:

默认拉取频率

scrape_interval: 15s

拉取超时时间

scrape_timeout: 10s

评估规则频率

evaluation_interval: 15s

规则文件配置

rule_files: [‘/etc/prometheus/rules/*.yml’]

告警配置

alerting:
alertmanagers:

• follow_redirects: true
  scheme: http
  timeout: 10s
  api_version: v2
  static_configs:
  • targets: []

拉取配置，添加监控项

scrape_configs:

监控prometheus

• job_name: prometheus
  metrics_path: /metrics
  static_configs:
  • targets:
    • localhost:9090

监控ip是否能ping通，docker启动的blackbox-exporter不建议用此监控，可能会有报错

• job_name: icmp_ping
  metrics_path: /probe
  params:
  module: [icmp]
  file_sd_configs:
  • files: [‘/apps/mon/prometheus-2.30.0.linux-amd64/dis/icmp_ping.yml’]
    refresh_interval: 10s
    relabel_configs:
  • source_labels: [address]
    regex: (.*)(:80)?
    target_label: __param_target
    replacement: ${1}
  • source_labels: [__param_target]
    target_label: instance
  • source_labels: [__param_target]
    regex: (.*)
    target_label: ping
    replacement: ${1}
  • source_labels: []
    regex: .*
    target_label: address
    replacement: localhost:9115

#icmp_ping.yml
位置：/etc/prometheus/conf.d/icmp_ping/icmp_ping.yml

• targets: [‘192.168.7.254’, ‘192.168.10.200’]
  labels:
  group: ‘ping监控’

---

监控端口是否能连通

• job_name: tcp_port
  metrics_path: /probe
  params:
  module: [tcp_connect]
  file_sd_configs:
  • files: [‘/apps/mon/prometheus-2.30.0.linux-amd64/dis/tcp_port.yml’]
    refresh_interval: 10s
    relabel_configs:
  • source_labels: [address]
    target_label: __param_target
  • source_labels: [__param_target]
    target_label: instance
  • target_label: address
    replacement: localhost:9115

#tcp_port.yml
位置：/etc/prometheus/conf.d/tcp_port/tcp_port.yml

• targets: [‘10.120.150.12:80’, ‘10.120.150.13:80’]
  labels:
  group: ‘Nginx-端口监控’

---

监控get请求

• job_name: http_get
  metrics_path: /probe
  params:
  module: [http_2xx]
  file_sd_configs:
  • files: [‘/etc/prometheus/conf.d/http_get/*.yml’]
    refresh_interval: 10s
    relabel_configs:
  • source_labels: [address]
    target_label: __param_target
  • source_labels: [__param_target]
    target_label: instance
  • target_label: address
    replacement: 192.168.7.254:9115

#http_get.yml
位置：/etc/prometheus/conf.d/http_get/http_get.yml

• targets:

  • http://192.168.7.254:8082/api/heartbeat/check/
    labels:
    name: ‘get测试’

• targets:

  • http://192.168.7.254:8083/api/heartbeat/check_test/
    labels:
    name: ‘get测试2’

---

监控post请求

• job_name: http_post
  metrics_path: /probe
  params:
  module: [http_post_2xx]
  file_sd_configs:
  • files: [‘/etc/prometheus/conf.d/http_post/*.yml’]
    refresh_interval: 10s
    relabel_configs:
  • source_labels: [address]
    target_label: __param_target
  • source_labels: [__param_target]
    target_label: instance
  • target_label: address
    replacement: 192.168.7.254:9115

#监控post请求

• job_name: http_post_with_token
  metrics_path: /probe
  params:
  module: [http_post_2xx_with_prometheus_post_check_token]
  file_sd_configs:
  • files: [‘/etc/prometheus/conf.d/http_post_with_token/*.yml’]
    refresh_interval: 10s
    relabel_configs:
  • source_labels: [address]
    target_label: __param_target
  • source_labels: [__param_target]
    target_label: instance
  • target_label: address
    replacement: 192.168.7.254:9115

#http_post.yml
位置：/etc/prometheus/conf.d/http_post/http_post.yml

• targets:
  • http://192.168.7.254:8082/api/heartbeat/post_check/
    labels:
    name: ‘post测试’

#http_post_with_token.yml
位置：/etc/prometheus/conf.d/http_post_with_token/http_post_with_token.yml

• targets:
  • http://192.168.7.254:8082/api/heartbeat/post_check/
    labels:
    name: ‘post带body测试’

---

#规则
rules.yml
位置：/etc/prometheus/rules/rules.yml

groups:

• name: probe_http_status_code
  rules:
  • alert: probe_http_status_code
    expr: probe_http_status_code != 200
    for: 1m
    labels:
    severity: critical
    annotations:
    summary: “{{ $labels.instance }} 状态码异常”
    description: “请尽快检测”
    groups:
• name: probe_success
  rules:
  • alert: probe_success
    expr: probe_success == 0
    for: 1m
    labels:
    severity: critical
    annotations:
    summary: “接口/主机/端口 {{ $labels.instance }} 无法联通”
    description: “请尽快检测”

#模板ID 9965号模板，数据源选择Prometheus 模板下载地址
https://grafana.com/grafana/dashboards/9965