Skip to content

pgo monitoring enabled but the monitoring user not created inside postgres and the metrics fail to generate #4501

Description

@sinawic

Overview

Environment

  • Platform: (Kubernetes)
  • Platform Version: (v1.31.1)
  • PGO Image Tag: (ubi9-6.0.1-0)
  • Postgres Version (18)
  • Storage: (local-path)

Steps to Reproduce

git clone https://github.com/CrunchyData/postgres-operator.git
cd postgres-operator
git checkout v6.0.1
kubectl apply -k config/namespace
kubectl apply --server-side -k config/default

the config i used to create my cluster using kustomize

apiVersion: postgres-operator.crunchydata.com/v1
kind: PostgresCluster
metadata:
  name: sinawic
spec:
  service:
    type: NodePort
    nodePort: 30432
  postgresVersion: 18
  users:
    - name: postgres
      databases:
        - postgres
  instances:
    - name: instance1
      replicas: 2
      dataVolumeClaimSpec:
        accessModes:
          - "ReadWriteOnce"
        storageClassName: local-path
        resources:
          requests:
            storage: 8Gi
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/hostname
                    operator: In
                    values:
                      - node2
                      - node3

  patroni:
    dynamicConfiguration:
      postgresql:
        parameters:
          max_connections: 100
          idle_in_transaction_session_timeout: 60000
          idle_session_timeout: 60000

  monitoring:
    pgmonitor:
      exporter: {}

EXPECTED

the exporter container cant seem to be able to connect to the postgres instance, and as far as i went through, the user ccp_monitoring doesnt get created and it seems its exact user the exporter is trying to connect to postgres with.
so i inspected the exporter metrics. it responds but the metrics are just general things like related to go and stuff like that.
the password secret for the ccp_monitoring user is created and injected int the exporter container but not the user itself.

after i created and granted the required user:

CREATE USER ccp_monitoring WITH PASSWORD 'password_created_for_monitoring_user_in_the_monitoring_secret';
GRANT pg_monitor TO ccp_monitoring;

metrics got generated and apparently things are back to normal.
but i expected by doing:

  monitoring:
    pgmonitor:
      exporter: {}

it should have been automatically created, like in the previous version <16>.
i followed the official docs in https://access.crunchydata.com/documentation/postgres-operator/latest/architecture/monitoring
and it states that we should either enable the metrics or the otel.

Logs

level=INFO source=server.go:73 msg="Established new database connection" fingerprint=localhost:5432
level=ERROR source=collector.go:178 msg="Error opening connection to database" err="error querying postgresql version: pq: password authentication failed for user \"ccp_monitoring\""
level=ERROR source=postgres_exporter.go:684 msg="error scraping dsn" err="Error opening connection to database (postgresql://ccp_monitoring:PASSWORD_REMOVED@localhost:5432/postgres): pq: password authentication failed for user \"ccp_monitoring\"" dsn=postgresql://ccp_monitoring:PASSWORD_REMOVED@localhost:5432/postgres

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions