Skip to content

Add ticket-writing and work-planning skills (PROF-16542) - #960

Merged
rkennke merged 11 commits into
mainfrom
rkennke/PROF-16542-planning-skills
Nov 11, 2026
Merged

rkennke merged 11 commits into
mainfrom
rkennke/PROF-16542-planning-skills

Conversation

@rkennke

@rkennke rkennke commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

Adds two agent skills for the ticket and planning conventions, which the repo did not record anywhere until now:

  • .agents/skills/ticket-writing/:
    • ticket shape and fields: parent Task and Sub-task, component, explicit assignee;
    • description sections;
    • sign-off and evidence comments, with links instead of bare names;
    • the status that each phase of the work maps to.
  • .agents/skills/work-planning/:
    • design: decisions with a recommendation, environments, critical areas, rollout and recovery;
    • acceptance criteria tagged local, test-drive or production-check;
    • a Build plan that maps each criterion to a check.

AGENTS.md Work Tracking points to both skills. doc/WorkTracking.md points to them and allows the design to live in the ticket description.

Stacked on #959, which adds the check-skills.py gate that these skills pass.

Motivation:

These conventions lived only in the agent-configuration proposal and in how PROF-16469 and PROF-16542 were run. A fresh session did not know them. On main, a headless session asked to draft a sub-task left it "Unassigned", which makes Jira assign the component default, another engineer. Asked about a merged PR with production checks pending, it answered "In Progress". Step 2 of PROF-16466; design on PROF-16542.

Additional Notes:

  • Reviewer acknowledgment needed: these status rules are new team conventions, not a restatement of existing text:
    • a merged PR with production checks pending stays In Review;
    • the ticket stays In Progress while a planned PR is still to be written;
    • waiting for the engineer's sign-off is not Blocked;
    • a parent ticket follows its child tickets.
  • Ticket layout by judgment: ticket-writing does not prescribe a structure per size. A small change may live in one ticket with one PR; a large one may use a parent with sub-tickets that each carry their own design, criteria and plan.
  • Human review in the plan and the PR: work-planning asks which human reviews a change requires or recommends, with reasons; pr-writing gains one line to state that in PR descriptions.
  • Jira only: the skills and the AGENTS.md pointer do not mention GitHub issues for external contributors. Older mentions in AGENTS.md Work Tracking and doc/WorkTracking.md are unchanged.
  • doc/WorkTracking.md used to place the design in a draft PR. It now allows the ticket description as the one declared source of truth, matching how these tickets are run.
  • Not in this PR: Jira automations for the status moves (PROF-16543).
  • Pre-PR review: /code-review (medium) on the first commit found 10 issues: disagreements with doc/WorkTracking.md, unclear status rules, and duplicated rules. The second commit fixes all 10. /code-review (low) on the fix found nothing. No Sphinx run, because the PR changes only Markdown.
  • Bot review follow-ups:
    • Codex and Bits found that work-planning forced the design and plan into the ticket, while doc/WorkTracking.md allows other locations. The skill now keeps them in the ticket unless the engineer chooses a location WorkTracking.md allows. Committing a plan or design file needs permission.
    • Codex found that ticket-writing let old evidence stand after a candidate change. A check is now re-run, and until then it is inconclusive, naming the earlier commit and its result.
    • Three /code-review rounds on these fixes kept finding new wording edges. The rounds stopped under the Add review depth, self-check and repeated-failure steps to pre-pr-review (PROF-16542) #962 stop rule.
  • Known, not fixed: whether a sign-off names the accepted revision of an artifact kept outside the ticket.
  • Human review: recommended. The PR changes AGENTS.md, which every agent session loads, and sets new team conventions for tickets and planning.
  • Accepted cost: any candidate change, including a comment fix, needs the checks re-run before they are reported as passed.

How to test the change?:

Check State
python3 utils/check-skills.py passed, 5 skills
bash utils/tests/test_quality_check_scripts.sh passed, 231 groups
Claude Code 2.1.319 and Pi 1.2.0, headless, read-only tools and no MCP servers, prompts that do not name a skill, on 0355d2d passed: the ticket, status and Build-plan prompts load ticket-writing and work-planning in both harnesses
Same prompts on main vs. this branch passed: drafts set library-java and an explicit assignee (on main: unassigned, placeholder component); the status answer is In Review (on main: In Progress); Build plans are a criteria table of about 370–460 words (on main: 892 and 1375 words)

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16542

🤖 Generated with Claude Code

rkennke and others added 2 commits November 9, 2026 17:42
The ticket and planning conventions were recorded nowhere in the repo:
only in the agent-configuration proposal and in how the PROF-16469 and
PROF-16542 tickets were run. Two skills now carry them and load at the
moment they apply.

- ticket-writing: parent Task and Sub-task shape, component, explicit
  assignee (Jira otherwise assigns the component default), description
  sections, sign-off and evidence comments, links instead of bare names,
  and the status each phase maps to. A merged PR with production checks
  pending stays In Review.
- work-planning: effort proportional to size, decisions listed with a
  recommendation, the environments a design must name, acceptance
  criteria tagged local, test-drive or production-check, and a Build plan
  that maps criteria to checks without restating the design.

AGENTS.md Work Tracking points to both skills.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es (PROF-16542)

- work-planning names the ticket description as the one declared source
  of truth, takes the critical areas from WorkTracking.md instead of a
  shorter copy, and asks for a rollout and recovery plan.
- ticket-writing evidence names the configuration that produced it, and
  for production checks the deployed revision, environment and window.
- Status: the ticket stays In Progress while a planned PR is still to be
  written; waiting for the engineer's sign-off is not Blocked; a parent
  Task follows its sub-tasks.
- Drop the rules AGENTS.md already keeps loaded (criterion sign-off,
  evidence states).
- WorkTracking.md allows the design in the ticket description and points
  to both skills; the AGENTS.md pointer covers GitHub issues and links
  the skill files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervmwuk1x
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 21.1.8 (6ubuntu1)
Date:Tue Nov 10 16:48:31 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs2
Logic error
Dereference of null pointer1
Unused code
Dead initialization1

Filter Results by File

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead initializationreferenceChains.cppapplyOomRamp4971
Logic errorDereference of null pointerperfEvents_linux.cppunregisterThread72026

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Pipelines

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d275af7 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #42857976983 | Commit: 98f576c | Duration: 21m 8s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-11-10 17:11:46 UTC

@rkennke
rkennke marked this pull request as ready for review November 9, 2026 20:44
@rkennke
rkennke requested a review from a team as a code owner November 9, 2026 20:44
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-09T20:50:38.087320Z 0355d2d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

The new mandatory skill in SKILL.md requires both the design and Build-plan artifacts to live in the ticket description, directly conflicting with doc/WorkTracking.md, which explicitly permits those artifacts to reside in a draft PR or a linked plan.md. The skill's amendment instruction also risks leaving the accepted plan stale or creating duplicate copies when the canonical source is outside the ticket.

Open Bits AI session

🤖 Bits Code Review · Commit 0355d2d

Comment thread .agents/skills/work-planning/SKILL.md Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0355d2d6db

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread .agents/skills/ticket-writing/SKILL.md Resolved
Comment thread .agents/skills/work-planning/SKILL.md Resolved
rkennke and others added 6 commits November 9, 2026 21:43
…542)

Codex and Bits review findings on the planning skills:

- work-planning forced the design and Build plan into the ticket
  description. doc/WorkTracking.md also allows a draft PR for the design
  and a versioned plan.md for the plan. The skill now keeps each artifact
  in its one declared source of truth, with the ticket as the default,
  and amends the plan there.
- ticket-writing let an agent keep old evidence after the candidate
  changed, given an explanation. AGENTS.md says a changed candidate
  invalidates evidence. Checks are now re-run on the new candidate;
  results from an earlier commit are reported as such.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(PROF-16542)

Review findings on the previous commit:

- Offering a "versioned plan.md" on equal footing conflicted with the
  AGENTS.md rule that plans are never committed without permission.
  The ticket description is now the default for the design, the
  acceptance criteria and the Build plan. A draft PR or plan.md is used
  only when the engineer chooses it, is linked from the ticket and the
  PR, and a plan.md is committed only with permission. ticket-writing's
  sub-task shape allows those links.
- Amendments cover the design as well as the plan.
- A result from an earlier commit is reported as inconclusive for the
  new candidate, naming its commit, until the check is re-run: one of
  the four evidence states, not prose.

Accepted cost: any candidate change, including a comment fix, needs the
checks re-run before they are reported as passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A stale passed result becomes inconclusive for a new candidate; a
  failed result stays failed, so a known failure is never hidden.
- Only the Design and Build plan may be links on a Sub-task; Intent and
  acceptance criteria stay on the ticket.
- A plan.md must be committed to be linkable, which needs the
  engineer's permission; without it the plan stays in the ticket. The
  draft-PR design is linked from the ticket, not from itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lusive (PROF-16542)

Each review round found a new conflict between the skill's own location
rules, doc/WorkTracking.md and the AGENTS.md rule against committing
plans. The skill now keeps the artifacts in the ticket description,
unless the engineer chooses another location WorkTracking.md allows, and
says that committing a plan or design file needs permission. It no
longer restates WorkTracking.md's options.

A check result from an earlier commit, passed or failed, is reported as
inconclusive for the new candidate, naming that commit and its result.
This follows the invariant that a changed candidate invalidates
evidence, without hiding an earlier failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The work-tracking process, the skills and the agent-configuration rules
follow the Datadog AI Native SDLC Playbook. AGENTS.md now links it from
the Work Tracking section, scoped to planning tracked work, changing the
process, and changing the agent configuration, so other tasks do not
open it.

The link is Datadog-internal. The text says so and that it is kept on
purpose: the repo is public so that customers can inspect the code that
runs in their processes, and it takes no outside contributions to the
process, so a link outsiders cannot open does no harm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…42)"

This reverts commit bd2396b. The link is an explicit exception to
keeping internal links out of the public repo, unrelated to the skills
in this PR, so it moves to its own PR under PROF-16466.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from rkennke/PROF-16542-writing-skills to main November 10, 2026 15:14

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The planning skills preserve the permitted artifact locations and require fresh evidence after candidate changes. The completed static review found no actionable conflicts with the tracking guidance.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 91ae9f0

@dd-octo-sts

dd-octo-sts Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 d275af75

rkennke and others added 3 commits November 10, 2026 16:42
…d review needs (PROF-16542)

- The repo takes no outside contributions, so ticket-writing and its
  AGENTS.md pointer no longer mention GitHub issues for external
  contributors.
- ticket-writing's Shape rule no longer prescribes a parent Task with a
  Sub-task per step. The layout fits the size of the work by judgment:
  one ticket and one PR for a small change, a parent with sub-tickets
  and several PRs for a large one.
- work-planning's design names the human reviews a change requires or
  recommends, with reasons; pr-writing repeats that under Additional
  Notes, because reviewers read the PR, not the ticket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/code-review findings on the previous commit:

- Sub-tickets again carry their own Design, acceptance criteria and
  Build plan, or link them where work-planning allows, so every step
  keeps criteria to verify.
- The status rule and the header line no longer assume a parent Task
  with Jira Sub-tasks: a parent ticket follows its child tickets, and
  the header names a parent only when there is one.
- work-planning no longer cites a second reviewer for
  .claude/settings.json as an example; no rule in the repo requires it.
- pr-writing's human-review line applies to descriptions only; without
  a recorded requirement, the agent states its own assessment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zhengyu123

Copy link
Copy Markdown
Contributor

Automated review of this PR found 9 items. All are consistency issues in the new skill docs, not code defects. Not yet verified line-by-line against the final text.

.agents/skills/ticket-writing/SKILL.md

  • L19: Sub-tickets may link their design "as the work-planning skill allows", but work-planning grants no such option. The two skills conflict on where sub-ticket planning lives.
  • L24: "Always set it explicitly" for the assignee, with no fallback when the session has no engineer identity.
  • L67 vs. L74: The Blocked row ("an external decision") overlaps with the rule that waiting on the engineer's decision is not Blocked.
  • L47: Sign-off does not require the accepted revision of a design kept outside the ticket (for example, in a draft PR).
  • L66 vs. L72 / L79: The In Review, In Progress, and merged-with-pending-checks states overlap. Precedence is not stated.
  • L26: The status names and components (library-java, openjdk) appear nowhere else in the repo, so the PROF workflow is assumed rather than checked.

AGENTS.md

  • L479: Says to load ticket-writing only before creating, updating, or moving a ticket. The skill also governs comments, so comment-only sessions may skip the re-run and evidence-link rules.

.agents/skills/work-planning/SKILL.md

  • L9: The Build plan lives in the ticket description by default, but doc/WorkTracking.md's Build section still requires a versioned plan.md (or ticket-linked equivalent). This PR does not update that text.
  • L31: Signal-handler code is added as a critical area, but it is not on doc/WorkTracking.md's mandatory-steering list, and the skill does not say whether steering is required for it.

@rkennke rkennke mentioned this pull request Nov 11, 2026
2 of 3 tasks
@rkennke
rkennke merged commit 8988f51 into main Nov 11, 2026
121 checks passed
@rkennke
rkennke deleted the rkennke/PROF-16542-planning-skills branch November 11, 2026 19:42
@github-actions github-actions Bot added this to the 1.53.0 milestone Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants