Feat: authdb tables cleanup

[HeloiseJoffe]

Closes DIRACGrid/diracx-charts#222

Changes:

• Add functions to cleanup expired refresh tokens and flows from the AuthDB
• Add tests for each cleanup function in the DB

The global cleanup function can be called with this kubernetes cronjob :

metadata:
  name: diracx-cleanup-authdb
  namespace: diracx
spec:
  schedule: "0 0 1 * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: cleanup
              image: ghcr.io/diracgrid/diracx/services:dev
              command:
                - /bin/sh
                - -c
                - |
                  /opt/conda/bin/python -c "
                  import asyncio
                  import os
                  from diracx.core.settings import AuthSettings
                  from diracx.db.sql import AuthDB
                  from diracx.logic.auth.token import cleanup_expired_data
                  import logging

                  logging.basicConfig(
                    level=logging.INFO
                  )

                  async def main():
                    db_url = os.getenv('DIRACX_DB_URL_AUTHDB')
                    settings = AuthSettings()
                    db = AuthDB(db_url)

                    async with db.engine_context():
                      async with db:
                        await cleanup_expired_data(db, settings)

                  asyncio.run(main())
                  "
              envFrom:
                - secretRef:
                    name: diracx-secrets
                - secretRef:
                    name: diracx-sql-connection-urls
                - secretRef:
                    name: diracx-dynamic-secrets
              volumeMounts:
                - name: keystore
                  mountPath: /keystore/jwks.json
                  subPath: jwks.json
          volumes:
            - name: keystore
              secret: 
                secretName: diracx-jwks
                items:
                  - key: jwks.json
                    path: jwks.json

Where could I add documentation to guide administrators in setting up the cronjob?

[read-the-docs-community]

Documentation build overview

📚 diracx | 🛠️ Build #31738700 | 📁 Comparing f143436 against latest (70bf3c7)

🔍 Preview build

Show files changed (1 files in total): 📝 1 modified | ➕ 0 added | ➖ 0 deleted

File	Status
admin/reference/env-variables/index.html	📝 modified

[aldbr]

I have not started to review it, but could you also open a PR in diracx-charts with the cron job and the documentation please?

[aldbr]

Minor comment: it might be clearer to mention the unit of max_validity in the docstring to avoid any ambiguous usage (the most expected guess if you don't read the code is that max_validity is expressed in seconds, at least in my mind)

[aldbr]

Same for max_retention by the way

[aldbr]

Same here, especially that you have 2 max_retention and they are not even in the same units.
To improve clarity, we could even always express max_validity and max_retention using the same units throughout the diracx-db layer to avoid any confusion.
What do you think?

[aldbr]

Suggested change

	async def clean_expired_refresh_token(
	async def clean_expired_refresh_tokens(

[aldbr]

To avoid string formatting if the log level is disabled.

Suggested change

	f"Deleted {expired_tokens} expired and {revoked_tokens} revoked refresh tokens"
	"Deleted %d expired and %d revoked refresh tokens", expired_tokens, revoked_tokens)

[aldbr]

Do you need a db-url parameter?
Can't you get the DB URL from an environment variable ($DIRACX_DB_URL_AUTHDB)?

[aldbr]

Minor comment but this could be split into 2 different functions to respect the single-responsibility principle: one function to delete expired refresh tokens, another one to delete old revoked refresh tokens.

[aldbr]

PENDING and READY flows never transition to a terminal state on their own from what I can see, do you confirm?
If one abandons the flow (stops polling, closes browser), the row stays forever.

I don't think we need any intermediate EXPIRED status because the status field is never consulted for expiration. The error messages are already returned based on time alone.

You should also delete PENDING/READY flows older than their max_retention time I think (note: we should make sure that max_retention is bigger than device/authorize_flow_expiration_seconds).

Any opinion?