Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/adapters/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ extension point = a documented, labelled slot with a tracking issue.
| [`tools/mail-source`](../../tools/mail-source/) | mbox, IMAP | Mailman 3 ([#306](https://github.com/apache/magpie/issues/306)) |
| [`tools/forwarder-relay`](../../tools/forwarder-relay/) | ASF-security ([`tools/gmail/asf-relay.md`](../../tools/gmail/asf-relay.md)) | huntr.com, HackerOne, GHSA relay |
| [`tools/scan-format`](../../tools/scan-format/) | ASVS | other scanner formats |
| [`tools/vcs`](../../tools/vcs/) | Git | Mercurial [#601](https://github.com/apache/magpie/issues/601), Subversion [#602](https://github.com/apache/magpie/issues/602), Jujutsu [#603](https://github.com/apache/magpie/issues/603), Fossil [#604](https://github.com/apache/magpie/issues/604), Perforce [#605](https://github.com/apache/magpie/issues/605) |
| Forge / tracker | [`github`](../../tools/github/), [`jira`](../../tools/jira/) | GitLab [#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [#310](https://github.com/apache/magpie/issues/310), Pagure [#312](https://github.com/apache/magpie/issues/312), Bitbucket [#606](https://github.com/apache/magpie/issues/606), SourceHut [#607](https://github.com/apache/magpie/issues/607), Bugzilla [#302](https://github.com/apache/magpie/issues/302) |
| [`tools/vcs`](../../tools/vcs/) | Git, Mercurial | Subversion [\#602](https://github.com/apache/magpie/issues/602), Jujutsu [\#603](https://github.com/apache/magpie/issues/603), Fossil [\#604](https://github.com/apache/magpie/issues/604), Perforce [\#605](https://github.com/apache/magpie/issues/605) |
| Forge / tracker | [`github`](../../tools/github/), [`jira`](../../tools/jira/), [`sourcehut`](../../tools/sourcehut/) | GitLab [\#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [\#310](https://github.com/apache/magpie/issues/310), Pagure [\#312](https://github.com/apache/magpie/issues/312), Bitbucket [\#606](https://github.com/apache/magpie/issues/606), Bugzilla [\#302](https://github.com/apache/magpie/issues/302) |
| Agentic runtime | Claude Code | Codex [#313](https://github.com/apache/magpie/issues/313)–OpenHands [#322](https://github.com/apache/magpie/issues/322) |
| Security cross-ref | — | OSV.dev [#311](https://github.com/apache/magpie/issues/311) |

Expand Down
3 changes: 2 additions & 1 deletion docs/labels-and-capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,8 @@ it implements multiple contracts (e.g. `tools/gmail` provides both
| [`tools/spec-validator`](../tools/spec-validator/) | `substrate:framework-dev` | Spec-frontmatter and body-section validator — counterpart to `skill-and-tool-validator` for `tools/spec-loop/specs/` |
| [`tools/symlink-lint`](../tools/symlink-lint/) | `substrate:framework-dev` | Self-adoption symlink hygiene — rejects cyclic symlinks and misdirected skill relays (canonical/relay target-correctness) |
| [`tools/pilot-report-validator`](../tools/pilot-report-validator/) | `substrate:framework-dev` | Adopter pilot-report validator — required frontmatter keys, no unfilled placeholders, valid profile, and required body sections; counterpart to `spec-validator` for `docs/pilot-report-template.md` |
| [`tools/vcs`](../tools/vcs/) | `contract:source-control` | Backend-dispatching implementation of the source-control (VCS) capability ([`tools/github/source-control.md`](../tools/github/source-control.md)); complete Git backend plus detected extension points for non-Git VCS bridges (#601 Hg, #602 SVN) |
| [`tools/vcs`](../tools/vcs/) | `contract:source-control` | Backend-dispatching implementation of the source-control (VCS) capability ([`tools/github/source-control.md`](../tools/github/source-control.md)); complete Git and Mercurial (Hg) backends, plus detected extension point for SVN (#602) |
| [`tools/sourcehut`](../tools/sourcehut/) | `contract:tracker` + `contract:source-control` + `contract:mail-archive` | SourceHut (sr.ht) forge bridge: todo.sr.ht, lists.sr.ht, builds.sr.ht, and git/hg repository reads |

A tool's capability is the **interface it provides**, not which skills
happen to consume it (RFC-AI-0005). `tools/github` provides the
Expand Down
24 changes: 12 additions & 12 deletions docs/vendor-neutrality.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ with pluggable backends already include:
| [`tools/mail-source`](../tools/mail-source/) | mbox, IMAP, Gmail API ([`tools/gmail`](../tools/gmail/)), Mailman 3 |
| [`tools/forwarder-relay`](../tools/forwarder-relay/) | ASF Security relay, huntr.com, HackerOne triagers |
| [`tools/scan-format`](../tools/scan-format/) | security-scanner report formats (ASVS reference) |
| [`tools/vcs`](../tools/vcs/) | Git (complete), Mercurial, Subversion, … (extension points) |
| [`tools/vcs`](../tools/vcs/) | Git (complete), Mercurial (complete), Subversion, … (extension points) |

The security-team surface follows the same pattern: CNA backends live
behind [`tools/cve-tool`](../tools/cve-tool/) (the ASF Vulnogram adapter
Expand Down Expand Up @@ -393,10 +393,9 @@ fixed that:
(`magpie-vcs`) runs the *abstract* operation and detects the active
backend from the working copy.

Today: **Git is complete** (the default binding); Mercurial
([#601](https://github.com/apache/magpie/issues/601)) and Subversion
([#602](https://github.com/apache/magpie/issues/602)) are real, detected
extension points that raise an actionable error naming their tracking
Today: **Git and Mercurial are complete** (the Git and Mercurial bindings); Subversion
([#602](https://github.com/apache/magpie/issues/602)) is a real, detected
extension point that raises an actionable error naming its tracking
issue until the full binding lands. Adding a backend means replacing one
`_UnimplementedBackend` with a concrete `VCSBackend` subclass —
detection, dispatch, the CLI, and every skill that calls `magpie-vcs`
Expand All @@ -411,8 +410,7 @@ GitHub-hosted ASF project that uses Git for source control needs
`tools/asf-svn` to steward its release flow through `dist.apache.org`.

Tracking issues exist, labelled `good first issue`, for the remaining
non-Git systems:
[Mercurial](https://github.com/apache/magpie/issues/601),
non-Git/non-Hg systems:
[Subversion](https://github.com/apache/magpie/issues/602) (generic VCS
binding; `tools/asf-svn` covers the full ASF SVN surface including
`dist.apache.org` and authorization),
Expand Down Expand Up @@ -484,9 +482,9 @@ coverage without pretending one team can implement an open-ended set.
|---|---|---|---|
| LLM backend | ✅ by construction | Claude Code, Ollama, vLLM, Apache-hosted, Bedrock, direct Anthropic | Any endpoint meeting the capability floor + privacy gate |
| Agentic runtime | ✅ by construction (`AGENTS.md` standard) | Claude Code; community use under Codex, Cursor, Gemini CLI, Copilot, OpenCode, Kiro | Runtime adapters [#313–#322](https://github.com/apache/magpie/issues?q=is%3Aissue+state%3Aopen+adapter+in%3Atitle) |
| Forge / tracker | ✅ by construction | GitHub, Jira; CVE/scan/relay via adapter contracts | GitLab [#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [#310](https://github.com/apache/magpie/issues/310), Pagure [#312](https://github.com/apache/magpie/issues/312), Bitbucket [#606](https://github.com/apache/magpie/issues/606), SourceHut [#607](https://github.com/apache/magpie/issues/607), Bugzilla [#302](https://github.com/apache/magpie/issues/302) |
| Forge / tracker | ✅ by construction | GitHub, Jira, SourceHut; CVE/scan/relay via adapter contracts | GitLab [#305](https://github.com/apache/magpie/issues/305), Forgejo/Gitea [#310](https://github.com/apache/magpie/issues/310), Pagure [#312](https://github.com/apache/magpie/issues/312), Bitbucket [#606](https://github.com/apache/magpie/issues/606), Bugzilla [#302](https://github.com/apache/magpie/issues/302) |
| Communication channels | ✅ by construction | PonyMail / mail-archive reads | mbox [#304](https://github.com/apache/magpie/issues/304), IMAP [#303](https://github.com/apache/magpie/issues/303), Mailman 3 [#306](https://github.com/apache/magpie/issues/306); Discourse [#307](https://github.com/apache/magpie/issues/307), Zulip [#308](https://github.com/apache/magpie/issues/308), Matrix [#309](https://github.com/apache/magpie/issues/309) |
| Source control (VCS) | ✅ by construction | **Git (complete)**; ASF SVN surface ([`tools/asf-svn`](../tools/asf-svn/): source control + dist.apache.org + authorization) | Mercurial [#601](https://github.com/apache/magpie/issues/601), Subversion generic VCS binding [#602](https://github.com/apache/magpie/issues/602) (detected); Jujutsu [#603](https://github.com/apache/magpie/issues/603), Fossil [#604](https://github.com/apache/magpie/issues/604), Perforce [#605](https://github.com/apache/magpie/issues/605) (tracked) |
| Source control (VCS) | ✅ by construction | **Git (complete)**, **Mercurial (complete)**; ASF SVN surface ([`tools/asf-svn`](../tools/asf-svn/): source control + dist.apache.org + authorization) | Subversion generic VCS binding [\#602](https://github.com/apache/magpie/issues/602) (detected); Jujutsu [\#603](https://github.com/apache/magpie/issues/603), Fossil [\#604](https://github.com/apache/magpie/issues/604), Perforce [\#605](https://github.com/apache/magpie/issues/605) (tracked) |
| Project governance | ✅ by construction | ASF + non-ASF adopter profiles | Adopter config (modes, thresholds) |

✅ "by construction" means the workflows carry no vendor assumption;
Expand Down Expand Up @@ -544,14 +542,15 @@ approval on endpoint identity rather than vendor. Both appear in the
generated block below.

<!-- BEGIN vendor-neutrality-score — generated by `uv run --project tools/vendor-neutrality-score vendor-neutrality-score --markdown`; do not edit by hand -->

**Overall vendor-neutrality score: 10/10 capability contracts (100%).** Generated by [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/); re-run it to refresh this section.

| Capability contract | Neutral? | Class | Backends today | Basis |
|---|---|---|---|---|
| `contract:tracker` | ✅ | vendor-backed | Atlassian, GitHub | 2 backend vendors: Atlassian, GitHub |
| `contract:source-control` | ✅ | vendor-backed | Git, GitHub, Subversion | 3 backend vendors: Git, GitHub, Subversion |
| `contract:tracker` | ✅ | vendor-backed | Atlassian, GitHub, SourceHut | 3 backend vendors: Atlassian, GitHub, SourceHut |
| `contract:source-control` | ✅ | vendor-backed | Git, GitHub, SourceHut, Subversion | 4 backend vendors: Git, GitHub, SourceHut, Subversion |
| `contract:change-request` | ✅ | vendor-backed | Atlassian, GitHub, email | 3 backend vendors: Atlassian, GitHub, email |
| `contract:mail-archive` | ✅ | vendor-backed | Google, PonyMail | 2 backend vendors: Google, PonyMail |
| `contract:mail-archive` | ✅ | vendor-backed | Google, PonyMail, SourceHut | 3 backend vendors: Google, PonyMail, SourceHut |
| `contract:mail-source` | ✅ | vendor-backed | Google, Maildir, PonyMail | 3 backend vendors: Google, Maildir, PonyMail |
| `contract:mail-create` | ✅ | vendor-backed | Google, Maildir | 2 backend vendors: Google, Maildir |
| `contract:cve-authority` | ✅ | vendor-backed | CVE.org, Vulnogram | 2 backend vendors: CVE.org, Vulnogram |
Expand Down Expand Up @@ -612,6 +611,7 @@ Harness → substrate tools it supports:
| Air-gapped on-prem | A PMC-hosted inference appliance on a private VLAN |

Every other endpoint is **opt-in** — the adopting project's security team declares it in `<project-config>/privacy-llm.md` (endpoint URL, data-residency contract, approver), so the choice is local and audited.

<!-- END vendor-neutrality-score -->

### What the number means
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -119,4 +119,5 @@ members = [
"tools/symlink-lint",
"tools/vcs",
"tools/vendor-neutrality-score",
"tools/sourcehut",
]
55 changes: 55 additions & 0 deletions tools/sourcehut/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [SourceHut (sr.ht) Forge Bridge](#sourcehut-srht-forge-bridge)
- [Prerequisites](#prerequisites)
- [Features](#features)
- [Invocation](#invocation)
- [Configuration](#configuration)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

# SourceHut (sr.ht) Forge Bridge

**Capability:** contract:tracker + contract:source-control + contract:mail-archive
Comment thread
onlyarnav marked this conversation as resolved.
**Kind:** implementation
**Vendor:** SourceHut

SourceHut (sr.ht) forge bridge implementation for the Apache Magpie framework. It integrates ticket tracking (`todo.sr.ht`), mailing list patchset review (`lists.sr.ht`), CI builds (`builds.sr.ht`), and repository reads (`git.sr.ht` & `hg.sr.ht`) using SourceHut's GraphQL APIs.

## Prerequisites

- **Runtime:** Python 3.11+ run via `uv` (stdlib-only, no third-party package dependencies at runtime).
- **CLIs:** `git` (for Git repo interactions) and `hg` (for Mercurial repo interactions).
- **Credentials / auth:** `SRHT_TOKEN` environment variable containing a SourceHut Personal Access Token (OAuth2 bearer token) with appropriate scopes (e.g. `TICKETS:RW`, `LISTS:R`, `BUILDS:R`, `REPOS:R`).
- **Network:** Reaches `todo.sr.ht`, `lists.sr.ht`, `builds.sr.ht`, `git.sr.ht`, and `hg.sr.ht` endpoints over HTTPS (`/query`).

## Features

1. **VCS Repositories:** Reads repo metadata across `git.sr.ht` and `hg.sr.ht`.
2. **Issue Tracker:** Read/write operations (create ticket, comment, resolve status, update labels) on `todo.sr.ht` trackers.
3. **Mailing Lists:** Reads patchsets and threads from `lists.sr.ht`, mapping them to the uniform PR/MR review abstraction.
4. **CI Builds:** Reads job statuses from `builds.sr.ht`.
5. **GraphQL client:** Unified command line tool to execute arbitrary queries/mutations across sr.ht subdomains.

## Invocation

```bash
# Get ticket details
uv run --project tools/sourcehut magpie-sourcehut ticket get ~user/tracker-name 123

# Create comment on a ticket
uv run --project tools/sourcehut magpie-sourcehut ticket comment ~user/tracker-name 123 --body "Nice fix!"

# Check build status
uv run --project tools/sourcehut magpie-sourcehut build get 123456
```

## Configuration

The bridge is configured via environment variables:

| Variable | Description |
|---|---|
| `SRHT_TOKEN` | Required. SourceHut personal OAuth2 token with access to target repositories, trackers, and mailing lists. |
80 changes: 80 additions & 0 deletions tools/sourcehut/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "magpie-sourcehut"
version = "0.1.0"
description = "SourceHut (sr.ht) forge bridge for Apache Magpie — provides tracker, patch review, VCS and CI status reads via GraphQL APIs."
readme = "README.md"
requires-python = ">=3.11"
license = { text = "Apache-2.0" }
dependencies = []

[project.scripts]
magpie-sourcehut = "magpie_sourcehut:main"

[tool.hatch.build.targets.wheel]
packages = ["src/magpie_sourcehut"]

[tool.ruff]
line-length = 110
target-version = "py311"
src = ["src", "tests"]

[tool.ruff.lint]
select = [
"E",
"W",
"F",
"I",
"B",
"UP",
"SIM",
"C4",
"RUF",
]
ignore = [
"E501",
]

[tool.ruff.lint.per-file-ignores]
"tests/**" = ["B", "SIM"]

[tool.mypy]
python_version = "3.11"
files = ["src", "tests"]
warn_unused_ignores = true
warn_redundant_casts = true
warn_unreachable = true
check_untyped_defs = true
no_implicit_optional = true
disallow_untyped_defs = true
disallow_incomplete_defs = true

[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false
disallow_incomplete_defs = false

[tool.pytest.ini_options]
minversion = "8.0"
addopts = "-ra -q"
testpaths = ["tests"]
36 changes: 36 additions & 0 deletions tools/sourcehut/src/magpie_sourcehut/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

"""SourceHut forge bridge implementation for Apache Magpie."""

from __future__ import annotations

import sys
from collections.abc import Sequence

__all__ = ["main"]


def main(argv: Sequence[str] | None = None) -> int:
"""CLI entry point."""
from magpie_sourcehut.cli import main as cli_main

try:
return cli_main(argv)
except Exception as exc:
print(f"magpie-sourcehut error: {exc}", file=sys.stderr)
return 1
49 changes: 49 additions & 0 deletions tools/sourcehut/src/magpie_sourcehut/builds.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

"""builds.sr.ht build integration."""

from __future__ import annotations

from typing import Any

from magpie_sourcehut.client import query_graphql


def get_job(job_id: int) -> dict[str, Any]:
"""Retrieve details of a specific job on builds.sr.ht."""
q = """
query GetJob($id: Int!) {
job(id: $id) {
id
status
created
updated
note
tags
visibility
image
runner
tasks {
name
status
}
}
}
"""
res = query_graphql("builds", q, {"id": job_id})
return res.get("job") or {}
Loading
Loading