Fix benchmark manifest ops CI generation by Copilot · Pull Request #51 · githubnext/apm

Copilot · 2026-05-15T03:42:43Z

Description

scripts/benchmark_manifest_ops.py could not reliably run in sandboxed environments, blocking CI-generated benchmark results for the public docs site.

Make benchmark temp-tree creation use an explicit writable --work-dir
Add markdown/doc update modes for regenerating the manifest benchmark table
Run the benchmark in the docs workflow before building GitHub Pages
Replace the stale docs note with an MDX-safe generated benchmark block
Add focused tests for work-dir usage, cleanup, and generated block replacement

Example CI invocation:

python3 scripts/benchmark_manifest_ops.py \
  --work-dir "$RUNNER_TEMP" \
  --markdown \
  --update-doc docs/src/content/docs/progress/autoloop-go-migration.mdx

Type of change

Bug fix
New feature
Documentation
Maintenance / refactor

Testing

Tested locally
All existing tests pass
Added tests for new functionality (if applicable)

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.49 to 3.1.50. - [Release notes](https://github.com/gitpython-developers/GitPython/releases) - [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES) - [Commits](gitpython-developers/GitPython@3.1.49...3.1.50) --- updated-dependencies: - dependency-name: gitpython dependency-version: 3.1.50 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Install Autoloop

…op-freq-30-minutes Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Ensure all coding agents know to run 'gh aw compile' and 'apm compile' after modifying workflow files, preventing stale integration file CI failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add agentic workflow recompilation instruction

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

…ntegration-files Refresh stale APM integration files

…-minutes Set autoloop cadence to 30 minutes

Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0. - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@2.6.3...2.7.0) --- updated-dependencies: - dependency-name: urllib3 dependency-version: 2.7.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>

Bumps [mermaid](https://github.com/mermaid-js/mermaid) from 11.14.0 to 11.15.0. - [Release notes](https://github.com/mermaid-js/mermaid/releases) - [Commits](https://github.com/mermaid-js/mermaid/compare/mermaid@11.14.0...mermaid@11.15.0) --- updated-dependencies: - dependency-name: mermaid dependency-version: 11.15.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

chore(deps): bump gitpython from 3.1.49 to 3.1.50

Fix agentic workflow CI failures

…ermaid-11.15.0 chore(deps): bump mermaid from 11.14.0 to 11.15.0 in /docs

chore(deps): bump urllib3 from 2.6.3 to 2.7.0

Fixes ./actions/setup monorepo subtree ref that was breaking every workflow run. Includes bundle transport shallow checkout fix (#31603). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Recompile all lock files with gh-aw main

The copilot-requests permission isn't enabled on this repo, causing every workflow run to fail with "The permission 'copilot-requests' is not allowed". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Remove copilot-requests permission from autoloop

…and migrate 5 leaf modules - Set up Go module (github.com/githubnext/apm) - Migrate constants.py -> internal/constants (55 lines) - Migrate version.py -> internal/version (101 lines) - Migrate utils/short_sha.py -> internal/utils/sha (45 lines, with tests) - Migrate utils/paths.py -> internal/utils/paths (27 lines) - Migrate utils/normalization.py -> internal/utils/normalization (57 lines) - Create benchmarks/migration-status.json tracking 285/71696 lines (0.40%) - Add Go build targets to Makefile Run: https://github.com/githubnext/apm/actions/runs/25717987972 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ic_io, git_env to Go Run: https://github.com/githubnext/apm/actions/runs/25736801433 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…py to Go Run: https://github.com/githubnext/apm/actions/runs/25744614816 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…v.py and helpers.py to Go Run: https://github.com/githubnext/apm/actions/runs/25747630390 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… modules + migrate host_backends + policy/discovery Registered 6 previously untracked Go modules (+5033 lines): - deps/download_strategies (1122) -> internal/deps/downloadstrategies - deps/apm_resolver (918) -> internal/deps/apmresolver - core/operations (145) -> internal/core/operations - models/dependency/reference (1559) -> internal/models/depreference - primitives/discovery (612) -> internal/primitives/discovery - deps/plugin_parser (677) -> internal/deps/pluginparser New Go migrations (+1988 lines): - deps/host_backends (623) -> internal/deps/hostbackends: HostBackend interface + GitHubBackend/GHECloudBackend/GHESBackend sharing gitHubFamilyBase; ADOBackend/ GitLabBackend/GenericGitBackend; BackendFor/BackendForHost dispatch - policy/discovery (1365) -> internal/policy/discovery: DiscoverPolicy/ DiscoverPolicyWithChain; GitHub Contents API fetch; URL fetch; file load; hash-pin verification (sha256); JSON-based cache with TTL + stale fallback; minimal YAML policy parser; atomic cache writes via os.Rename Metric: 65.26 -> 75.06 (+9.80) Run: https://github.com/githubnext/apm/actions/runs/25886940959 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…flow Install Evergreen PR health workflow

[Autoloop: python-to-go-migration]

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

- Update latest status: iteration 51, PR #43, best metric 75.06% - Migrated lines: 53,813 of 71,696 (was 15,116) - Migrated modules: 141 (was 89) - Extend migration progress table with iters 36-51 - Replace migrated modules table with full current list from migration-status.json - Update benchmark signals and Go build/test validation (iters 49-51) - Update next-up section: github_downloader, mcp_integrator, agents_compiler - Timestamp: 2026-05-14 23:03 UTC Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…-migration-dashboard Replace docs site with migration progress dashboard

…evalue-5.8.1 chore(deps): bump devalue from 5.8.0 to 5.8.1 in /docs

…r51-a6f9d073544a8ec5 [autoloop-progress] docs: update Autoloop Go migration progress to iteration 51 (75.06%)

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

mrjf · 2026-05-21T05:20:11Z

@copilot fix ci

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Copilot · 2026-05-21T05:22:24Z

@copilot fix ci

Fixed the CI lint failure in e17e03a by applying the ruff formatting change to tests/scripts/test_benchmark_manifest_ops.py. Validated with uv run --extra dev ruff check src/ tests/, uv run --extra dev ruff format --check src/ tests/, and uv run --extra dev pytest tests/scripts/test_benchmark_manifest_ops.py.

mrjf · 2026-05-21T14:20:18Z

Closing as part of resetting githubnext/apm to match the current state of upstream microsoft/apm. The Python-to-Go migration experiment has been wound down. See commit ccdafc4 on main.

dependabot Bot and others added 30 commits May 9, 2026 08:28

Install Autoloop

61a6a75

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Initial plan

7cd3f7c

Merge pull request #2 from githubnext/install-autoloop

1bca27a

Install Autoloop

Initial plan

713ce57

Merge remote-tracking branch 'origin/main' into copilot/change-autolo…

4e543da

…op-freq-30-minutes Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Set autoloop schedule to 30 minutes

4212c46

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Fix agentic workflow CI failures

f4804bf

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Add agentic workflow recompilation instruction

8407338

Ensure all coding agents know to run 'gh aw compile' and 'apm compile' after modifying workflow files, preventing stale integration file CI failures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Merge pull request #8 from githubnext/add-recompile-instruction

4e5d009

Add agentic workflow recompilation instruction

Initial plan

5025124

Merge main into PR branch

8a4e023

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Refresh APM integration files

e53f567

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Merge pull request #10 from githubnext/copilot/rebase-and-recompile-i…

1168ea3

…ntegration-files Refresh stale APM integration files

Merge pull request #5 from githubnext/copilot/change-autoloop-freq-30…

f6f5272

…-minutes Set autoloop cadence to 30 minutes

Merge main and refresh autoloop lock

80e3d08

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Merge pull request #1 from githubnext/dependabot/uv/gitpython-3.1.50

8b7688b

chore(deps): bump gitpython from 3.1.49 to 3.1.50

Merge pull request #7 from githubnext/copilot/investigate-fix-ci-errors

7089449

Fix agentic workflow CI failures

Merge pull request #12 from githubnext/dependabot/npm_and_yarn/docs/m…

d02d49f

…ermaid-11.15.0 chore(deps): bump mermaid from 11.14.0 to 11.15.0 in /docs

Merge pull request #11 from githubnext/dependabot/uv/urllib3-2.7.0

6483245

chore(deps): bump urllib3 from 2.6.3 to 2.7.0

Recompile all lock files with gh-aw main

2fb7b01

Fixes ./actions/setup monorepo subtree ref that was breaking every workflow run. Includes bundle transport shallow checkout fix (#31603). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Merge pull request #14 from githubnext/fix/recompile-gh-aw-main

e17c175

Recompile all lock files with gh-aw main

Remove copilot-requests feature flag from autoloop

393ab4e

The copilot-requests permission isn't enabled on this repo, causing every workflow run to fail with "The permission 'copilot-requests' is not allowed". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Merge pull request #15 from githubnext/fix/remove-copilot-requests

c212afe

Remove copilot-requests permission from autoloop

[Autoloop: python-to-go-migration] Iteration 2: migrate yaml_io, atom…

4fea0e8

…ic_io, git_env to Go Run: https://github.com/githubnext/apm/actions/runs/25736801433 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

[Autoloop: python-to-go-migration] Iteration 3: migrate utils/guards.…

99e6ae1

…py to Go Run: https://github.com/githubnext/apm/actions/runs/25744614816 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

[Autoloop: python-to-go-migration] Iteration 4: Migrate subprocess_en…

9c40a5f

…v.py and helpers.py to Go Run: https://github.com/githubnext/apm/actions/runs/25747630390 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions Bot and others added 10 commits May 14, 2026 21:50

Merge pull request #41 from githubnext/copilot/install-evergreen-work…

ffa8f3a

…flow Install Evergreen PR health workflow

Merge pull request #43 from githubnext/autoloop/python-to-go-migration

4338c93

[Autoloop: python-to-go-migration]

Fix policy docs consistency tests

9bf04db

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

ci: trigger checks

e63e058

Merge pull request #46 from githubnext/copilot/replace-docs-site-with…

5c09d1d

…-migration-dashboard Replace docs site with migration progress dashboard

Merge pull request #47 from githubnext/dependabot/npm_and_yarn/docs/d…

c0c844f

…evalue-5.8.1 chore(deps): bump devalue from 5.8.0 to 5.8.1 in /docs

Merge pull request #48 from githubnext/docs/autoloop-go-migration-ite…

f7de14d

…r51-a6f9d073544a8ec5 [autoloop-progress] docs: update Autoloop Go migration progress to iteration 51 (75.06%)

Initial plan

7858c17

Copilot AI assigned Copilot and mrjf May 15, 2026

Copilot AI linked an issue May 15, 2026 that may be closed by this pull request

Fix benchmark_manifest_ops.py sandboxing #50

Closed

Copilot started work on behalf of mrjf May 15, 2026 03:42 View session

Copilot AI and others added 3 commits May 15, 2026 03:48

fix benchmark docs generation

3aea6cf

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

address benchmark review feedback

9b0d74d

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

polish benchmark docs helpers

279de78

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix benchmark_manifest_ops.py sandboxing issue~~ Fix benchmark manifest ops CI generation May 15, 2026

Copilot finished work on behalf of mrjf May 15, 2026 03:52

Copilot AI requested a review from mrjf May 15, 2026 03:52

mrjf marked this pull request as ready for review May 21, 2026 05:20

Copilot started work on behalf of mrjf May 21, 2026 05:20 View session

format benchmark manifest tests

e17e03a

Co-authored-by: mrjf <180956+mrjf@users.noreply.github.com>

Copilot finished work on behalf of mrjf May 21, 2026 05:22

mrjf force-pushed the main branch from 6fa1427 to ccdafc4 Compare May 21, 2026 14:18

mrjf closed this May 21, 2026

mrjf deleted the copilot/fix-benchmark-manifest-ops-sandboxing branch May 21, 2026 14:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix benchmark manifest ops CI generation#51

Fix benchmark manifest ops CI generation#51
Copilot wants to merge 115 commits into
mainfrom
copilot/fix-benchmark-manifest-ops-sandboxing

Copilot AI commented May 15, 2026 •

edited

Loading

Uh oh!

mrjf commented May 21, 2026

Uh oh!

Copilot AI commented May 21, 2026

Uh oh!

mrjf commented May 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Copilot AI commented May 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Testing

Uh oh!

mrjf commented May 21, 2026

Uh oh!

Copilot AI commented May 21, 2026

Uh oh!

mrjf commented May 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented May 15, 2026 •

edited

Loading