Skip to content

feat: add --format otel to scan command for ComplyBeacon evidence export#463

Open
gvauter wants to merge 7 commits intocomplytime:mainfrom
gvauter:003-complybeacon-export
Open

feat: add --format otel to scan command for ComplyBeacon evidence export#463
gvauter wants to merge 7 commits intocomplytime:mainfrom
gvauter:003-complybeacon-export

Conversation

@gvauter
Copy link
Copy Markdown
Member

@gvauter gvauter commented Apr 3, 2026

Summary

Implements the ComplyBeacon evidence export feature as specified in spec 003. This adds otel as a new output format for the complyctl scan command, enabling compliance evidence to be exported to a ComplyBeacon collector (or any OTLP-compatible endpoint) as OpenTelemetry log records.

When --format otel is specified, complyctl scan runs the scan as usual, then initiates an export phase where it calls each plugin's new Export RPC. The collector endpoint and optional OIDC client credentials are configured in complytime.yaml. Plugin stubs are included for OpenSCAP, AMPEL, and the test plugin — real ProofWatch-based implementations will follow in a subsequent PR.

Related Issues

Review Hints

Best reviewed commit-by-commit in order — each commit is self-contained and builds on the previous:

  • b562e48 proto — new Export RPC, CollectorConfig message, and supports_export field in the gRPC contract
  • 7fe9769 plugin SDK — domain types, client/server methods, and RouteExport in the manager
  • e602ef1 config — CollectorConfig and AuthConfig structs in complytime.yaml, plus the otel output format constant
  • f88be26 scan command — --format otel validation, OIDC token resolution, export orchestration, and summary table
  • 1188597 doctor — new CheckCollector diagnostic for the collector configuration
  • f5d1785 plugin stubs — Export method in OpenSCAP, AMPEL, and test plugins (returns "not yet supported")

Notes:
The scan command handles OIDC token exchange itself (via golang.org/x/oauth2/clientcredentials) and passes the resolved bearer token to plugins in the ExportRequest. Plugins never see client credentials directly.

All plugin Export implementations are currently stubs that return Success: false. A follow-up PR will implement real export using ProofWatch in one or more plugins.

To test locally: configure a collector section in complytime.yaml with an endpoint, then run complyctl scan --policy-id --format otel. Without a collector configured, the command returns a clear error message. complyctl doctor will also report the collector configuration status.

Assisted by Cursor AI

@gvauter gvauter self-assigned this Apr 3, 2026
@gvauter gvauter requested review from a team as code owners April 3, 2026 13:20
Copy link
Copy Markdown
Member

@jpower432 jpower432 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a PR with a suggestion to your branch. I would like to find a way to avoid plugin author needs to Export stubs if they don't implement it. I think this additional feature should be fully opt in for the plugin author.

gvauter#1

Comment thread api/plugin/plugin.proto

// Whether this plugin implements the Export RPC for shipping
// evidence to a Beacon collector. Defaults to false.
bool supports_export = 6;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this require a plugin to implement an empty Export? Should Export be a new interface?

@gvauter gvauter force-pushed the 003-complybeacon-export branch from 54f3f31 to 9a753d4 Compare April 15, 2026 23:03
@gvauter
Copy link
Copy Markdown
Member Author

gvauter commented Apr 15, 2026

I added a PR with a suggestion to your branch. I would like to find a way to avoid plugin author needs to Export stubs if they don't implement it. I think this additional feature should be fully opt in for the plugin author.

gvauter#1

Great thanks Jenn, I've merged your PR.

@gvauter gvauter force-pushed the 003-complybeacon-export branch from 9a753d4 to 2ae7d72 Compare April 15, 2026 23:41
@gvauter gvauter requested a review from jpower432 April 15, 2026 23:53
@gvauter gvauter force-pushed the 003-complybeacon-export branch from 2ae7d72 to 77c4c7d Compare April 16, 2026 00:28
gvauter and others added 5 commits April 15, 2026 20:55
Signed-off-by: George Vauter <[email protected]>
Made-with: Cursor
Export was part of the Plugin interface, forcing every plugin to
carry a no-op stub even when it has no export capability. This
violates the Interface Segregation Principle and contradicts
FR-013 ("plugins that do not implement Export MUST NOT be required
to change").

Introduce a separate Exporter interface. The grpcServer adapter
performs a runtime type assertion so the proto-level Export RPC
still works on the wire. Plugin authors only implement Exporter
when they opt in via supports_export=true in DescribeResponse.

Made-with: Cursor
Signed-off-by: Jennifer Power <[email protected]>
Made-with: Cursor
@gvauter gvauter force-pushed the 003-complybeacon-export branch from 77c4c7d to f9337b7 Compare April 16, 2026 00:56
marcusburghardt added a commit to marcusburghardt/org-infra that referenced this pull request Apr 16, 2026
The post-comment job in ci_crapload.yml was skipped when the CRAP Load
analysis detected regressions because the implicit success() condition
on the needs dependency prevented it from running. This meant PR authors
never saw the detailed report when they needed it most.

Add if: !cancelled() so the comment job runs on both success and failure.
Handle missing artifacts with a fallback comment linking to the Job
Summary, and truncate oversized comments to stay within GitHub limits.

Refs: complytime/complyctl#463

Assisted-by: OpenCode (claude-opus-4-6)
Signed-off-by: Marcus Burghardt <[email protected]>
marcusburghardt added a commit to complytime/org-infra that referenced this pull request Apr 16, 2026
The post-comment job in ci_crapload.yml was skipped when the CRAP Load
analysis detected regressions because the implicit success() condition
on the needs dependency prevented it from running. This meant PR authors
never saw the detailed report when they needed it most.

Add if: !cancelled() so the comment job runs on both success and failure.
Handle missing artifacts with a fallback comment linking to the Job
Summary, and truncate oversized comments to stay within GitHub limits.

Refs: complytime/complyctl#463

Assisted-by: OpenCode (claude-opus-4-6)
Signed-off-by: Marcus Burghardt <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement spec for export functionality

2 participants