feat: add session-profiling protocol and profile-session template per…#72
Conversation
Add profile-session template and session-profiling reasoning protocol for analyzing completed LLM session logs to identify token inefficiencies and structural waste. Built on the structure from PR microsoft#72 by @themechbro — keeps the 5-phase methodology and adds full PromptKit conventions: Protocol (session-profiling): - 5 phases: segment log, map to components, detect inefficiencies, quantify impact, produce recommendations - 7 inefficiency types: redundant reasoning, false starts, re-derivation, protocol loops, unused context, verbose compliance, persona drift - Each recommendation tied to a specific PromptKit component file Template (profile-session): - Full frontmatter: persona (specification-analyst), protocols (anti-hallucination, self-verification, session-profiling), format (investigation-report), params, contracts - Params: session_log, assembled_prompt, focus_areas - Non-goals: not a quality audit, not a guardrail remover Manifest: additive-only changes (protocol + template entries, no reformatting of existing content). Co-authored-by: themechbro <109350438+themechbro@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
b20c0d9 to
eeabaca
Compare
|
@themechbro — I pushed a co-engineering commit to your branch that builds on your 5-phase structure. Here's what changed: What I kept from your PR
What I addedProtocol — expanded each phase with the depth PromptKit protocols need (comparable to
Template — added full PromptKit frontmatter so the assembly engine can compose it:
Manifest — reverted the whitespace reformatting and kept only the 2 additive entries (protocol + template). The reformatting was causing 12 validation failures. For referenceCompare Let me know if you have questions or want to adjust anything! |
|
@themechbro — stepping back, I want to apologize. You asked for early feedback on your draft, and instead of reviewing and letting you iterate, I force-pushed a rewrite directly to your branch. That wasn't the right call — it's your PR and your learning opportunity, and I should have respected that. If you'd prefer to revert to your original commit and work from review feedback instead, that's completely fine. You can restore your version with: git reflog # find your original commit hash
git reset --hard <your-original-hash>
git push --forceEither way — your original structure (5 phases, the inefficiency categories, the file locations) was the right foundation. The main things that need to happen for the assembly engine to work with it are:
Happy to answer questions or do a more detailed review — whichever you'd find more helpful. Again, sorry for overstepping. |
|
@Alan-Jowett - I've pulled down your co-engineering commit and applied the fixes flagged by the Copilot review. Changes included in this push:
Let me know if this is ready to drop the "Draft" status! |
|
Thanks for sending this over — I really appreciate it. I haven’t forgotten about it; I just need a bit more time to look through it carefully. I’ll follow up once I’ve had a chance to dig in. |
|
Hey @themechbro — the protocol and template content here is looking good. The 5-phase structure, inefficiency categories (RE-REDUNDANT, RE-BACKTRACK, etc.), and the investigation-report integration all work well. The main thing blocking this is that the branch is based on an old version of To fix this, rebase onto current \\�ash After rebasing, your manifest.yaml diff should only contain:
If the diff still shows changes to personas, other protocols, or other templates, those are stale and need to be dropped. The rebase should handle this automatically if there are no conflicts. Once the diff is clean, this should be ready for a final review. Let me know if you hit any issues with the rebase! |
f4f1033 to
c0c1cbc
Compare
|
@Alan-Jowett — I’ve successfully rebased the branch onto the latest main. The diff is now clean and only contains:
Let me know if there’s anything else needed, or if this is ready for the final merge! |
|
Rebase looks clean — manifest diff is exactly the 2 additive entries (protocol + template), no stale changes. Content is solid. Go ahead and mark it ready for review when you're ready! |
There was a problem hiding this comment.
Pull request overview
Adds a new “session profiling” capability to PromptKit by introducing a dedicated reasoning protocol and a corresponding template, and wiring both into the component manifest so they can be assembled and used like other investigation workflows.
Changes:
- Added
session-profilingreasoning protocol for analyzing completed LLM session logs and attributing token waste to PromptKit components. - Added
profile-sessiontemplate that composes existing guardrails with the new protocol and outputs aninvestigation-report. - Updated
manifest.yamlto register the new protocol and template under the investigation category.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| templates/profile-session.md | New template for profiling an LLM session log and producing an investigation-report with optimization recommendations. |
| protocols/reasoning/session-profiling.md | New reasoning protocol defining phases for segmenting a session, attributing token usage, detecting inefficiencies, and recommending optimizations. |
| manifest.yaml | Registers session-profiling protocol and profile-session template so the CLI/assembler can resolve them. |
|
Heads up @themechbro — a few PRs are about to merge that add new components to manifest.yaml. You'll need another rebase before this can merge. The changes are in different manifest sections (personas, formats, new protocol categories) so it should auto-resolve cleanly: git fetch upstream
git rebase upstream/mainSorry for the churn — the library is growing fast this week! |
Add profile-session template and session-profiling reasoning protocol for analyzing completed LLM session logs to identify token inefficiencies and structural waste. Protocol (session-profiling): - 5 phases: segment log, map to components, detect inefficiencies, quantify impact, produce recommendations - 7 inefficiency types: redundant reasoning, false starts, re-derivation, protocol loops, unused context, verbose compliance, persona drift - Each recommendation tied to a specific PromptKit component file Template (profile-session): - Full frontmatter: persona (specification-analyst), protocols (anti-hallucination, self-verification, session-profiling), format (investigation-report), params, contracts - Params: session_log, assembled_prompt, focus_areas - Non-goals: not a quality audit, not a guardrail remover Manifest: additive-only changes (protocol + template entries, no reformatting of existing content). Co-authored-by: Alan-Jowett <alanjo@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
5dc9d81 to
9fc795f
Compare
@Alan-Jowett — All set! Rebased onto the latest main and force-pushed. It auto-resolved cleanly just like you suspected. Ready when you are! |
This is a fresh draft for the Session Profiler Template (Issue #44), scoped strictly to the template architecture as discussed in the previous PR.
Changes in this Draft:
@Alan-Jowett - Tagging you here for early feedback as requested! Let me know if the directory placement and YAML frontmatter look perfectly aligned with the engine's expectations before I build out the markdown/JSON formatting logic.