Skip to content

feat: support log line content filtering via RELABEL_CONFIGS#98

Open
Omerlustig wants to merge 1 commit intografana:mainfrom
Omerlustig:feat/relabel-drop-log-lines
Open

feat: support log line content filtering via RELABEL_CONFIGS#98
Omerlustig wants to merge 1 commit intografana:mainfrom
Omerlustig:feat/relabel-drop-log-lines

Conversation

@Omerlustig
Copy link

Summary

  • Enable RELABEL_CONFIGS to filter log entries based on log line content using a synthetic __log_line__ label, allowing users to drop noisy lines (e.g. END RequestId:) before they are sent to Loki
  • The __log_line__ label is injected temporarily before relabeling and stripped afterwards so it never reaches Loki
  • Fix a pre-existing bug where ScratchBuilder.Sort() was missing before relabel.Process, which uses binary search internally and produced non-deterministic results on unsorted labels

How it works

Users can now use the existing RELABEL_CONFIGS env var with source_labels: ["__log_line__"] to match against log line content:

[{"source_labels":["__log_line__"],"regex":"END RequestId:.*","action":"drop"}]

All Prometheus relabel actions (drop, keep, replace, labeldrop, etc.) are supported. Existing label-only relabel configs continue to work unchanged.

Test plan

  • TestBatchAddDropsLogLinesViaRelabelConfig — drop action on log line content
  • TestBatchAddKeepsEntriesWithNoRelabelConfig — no-op when no configs set
  • TestBatchAddKeepActionFiltersLogLines — keep action filtering
  • TestBatchAddLogLineLabelNotInOutput__log_line__ label never in output
  • TestBatchAddLabelOnlyRelabelStillWorks — backward compat with label-only relabel
  • TestBatchAddDropMultiplePatterns — multi-pattern drop
  • All 39 existing tests continue to pass

🤖 Generated with Claude Code

Enable RELABEL_CONFIGS to filter log entries based on log line content
using a synthetic __log_line__ label. This allows dropping noisy lines
(e.g. "END RequestId:") before they are sent to Loki, reducing ingestion
volume and cost.

The __log_line__ label is injected temporarily before relabeling and
stripped afterwards so it never reaches Loki. Also fixes a pre-existing
bug where labels were not sorted before relabel.Process (which uses
binary search), causing non-deterministic relabeling results.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants