-
Notifications
You must be signed in to change notification settings - Fork 5
Add simulation-validation protocol and validate-simulation template #100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Alan-Jowett
merged 4 commits into
microsoft:main
from
Alan-Jowett:add-simulation-validation
Mar 29, 2026
+340
−0
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
b47c329
Add simulation-validation protocol and validate-simulation template
4b4afe5
Fix Phase 6 category labels to match phase titles
da9d36b
Fix self-verification minimum and phase title examples
a23ab23
Mark voltage corner values as examples, not hard-coded
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,182 @@ | ||
| <!-- SPDX-License-Identifier: MIT --> | ||
| <!-- Copyright (c) PromptKit Contributors --> | ||
|
|
||
| --- | ||
| name: simulation-validation | ||
| type: analysis | ||
| description: > | ||
| Systematic review of circuit simulation output (SPICE, power budget | ||
| calculators, thermal analysis) against specification constraints. | ||
| Covers setup verification, result interpretation, constraint | ||
| compliance, corner-case coverage, and model validity assessment. | ||
| applicable_to: | ||
| - validate-simulation | ||
| --- | ||
|
|
||
| # Protocol: Simulation Validation | ||
|
|
||
| Apply this protocol when reviewing simulation output (SPICE transient/ | ||
| AC/DC analysis, power budget calculations, thermal simulations, or | ||
| other EDA tool results) against specification constraints. Execute all | ||
| phases in order. | ||
|
|
||
| ## Phase 1: Simulation Setup Review | ||
|
|
||
| Verify the simulation is configured to answer the right questions. | ||
|
|
||
| 1. **Identify the simulation type**: DC operating point, transient, | ||
| AC sweep, Monte Carlo, worst-case, thermal, or power budget. | ||
|
|
||
| 2. **Verify stimulus conditions**: Are the input stimuli (voltage | ||
| sources, current loads, clock frequencies, temperature) consistent | ||
| with the specification's operating conditions? | ||
| - Input voltage range matches spec (e.g., battery discharge curve, | ||
| USB voltage tolerance) | ||
| - Load profiles match the specified operating states (active, sleep, | ||
| burst) | ||
| - Temperature matches the specified operating range (not just room | ||
| temperature) | ||
|
|
||
| 3. **Verify component models**: | ||
| - Are SPICE models from the component manufacturer (not generic)? | ||
| - Are model parameters at the correct temperature/process corner? | ||
| - Are parasitic elements included where they matter (ESR for | ||
| capacitors, DCR for inductors, trace resistance for power paths)? | ||
| - Flag any missing models or placeholder components. | ||
|
|
||
| 4. **Verify measurement points**: Are the simulation probes measuring | ||
| the right nodes? A voltage measured at the regulator output is not | ||
| the same as voltage at the IC power pin (IR drop matters). | ||
|
|
||
| ## Phase 2: Result Extraction | ||
|
|
||
| Extract quantitative results from the simulation output. | ||
|
|
||
| 1. **For each measurement point**, extract: | ||
| - Steady-state value (DC operating point or settled value) | ||
| - Transient behavior (overshoot, undershoot, settling time, ringing) | ||
| - Ripple (peak-to-peak variation in steady state) | ||
| - Worst-case value across the simulation (min/max envelope) | ||
|
|
||
| 2. **For power analysis**, extract per-component and per-rail: | ||
| - Average power dissipation | ||
| - Peak power dissipation | ||
| - Current draw per operating state | ||
| - Total power budget per rail per state | ||
|
|
||
| 3. **For thermal analysis**, extract: | ||
| - Junction temperature per component | ||
| - Ambient-to-junction thermal path | ||
| - Thermal margin to absolute maximum rating | ||
|
|
||
| 4. **Record the units and conditions** for every extracted value. | ||
| A current measurement is meaningless without stating the operating | ||
| state, input voltage, and temperature at which it was taken. | ||
|
|
||
| ## Phase 3: Constraint Compliance Check | ||
|
|
||
| Compare each extracted result against the corresponding specification | ||
| constraint. | ||
|
|
||
| 1. **Map results to constraints**: For each specification constraint | ||
| (voltage tolerance, current limit, timing requirement, thermal | ||
| limit), identify the simulation result that addresses it. | ||
|
|
||
| 2. **Check compliance**: | ||
| - Does the simulated value satisfy the constraint under the | ||
| simulated conditions? | ||
| - Does it satisfy the constraint with adequate margin? (Use the | ||
| quantitative-constraint-validation methodology for margin | ||
| classification if formal margin analysis is needed.) | ||
|
|
||
| 3. **Flag non-compliance**: For each constraint that is not satisfied: | ||
| - What is the violation magnitude? | ||
| - Under what conditions does it occur (input voltage, load, temp)? | ||
| - Is the violation due to the design or due to simulation setup? | ||
|
|
||
| 4. **Flag uncovered constraints**: Specification constraints that have | ||
| no corresponding simulation result. These are coverage gaps — the | ||
| simulation doesn't answer whether the constraint is met. | ||
|
|
||
| ## Phase 4: Corner-Case and Worst-Case Assessment | ||
|
|
||
| Verify the simulation covers the corners that matter. | ||
|
|
||
| 1. **Input voltage corners**: Was the simulation run at minimum and | ||
| maximum input voltage, not just nominal? Use the actual voltage | ||
| ranges from the project's requirements/specification: | ||
| - Example: for a single-cell Li-ion battery, fully charged | ||
| (~4.2V) AND near-cutoff (~3.0V) | ||
| - Example: for USB 5V, minimum (~4.5V) AND maximum (~5.5V) | ||
Alan-Jowett marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - Flag if only nominal values were simulated | ||
|
|
||
| 2. **Load corners**: Was the simulation run at minimum and maximum | ||
| load? | ||
| - No load / sleep (leakage only) | ||
| - Maximum load (all peripherals active + radio TX) | ||
| - Load transients (sudden current step) | ||
|
|
||
| 3. **Temperature corners**: Was the simulation run across the | ||
| operating temperature range? | ||
| - Room temperature only is insufficient for production designs | ||
| - Component parameters (ESR, leakage, threshold voltage) vary | ||
| with temperature | ||
|
|
||
| 4. **Process corners** (if applicable): For IC-level simulations, | ||
| were fast/slow/typical corners included? | ||
|
|
||
| 5. **For each missing corner**, flag it as a coverage gap with the | ||
| specific risk: "Regulator dropout was only simulated at 25°C; | ||
| dropout voltage increases at high temperature per datasheet | ||
| Figure X." | ||
|
|
||
| ## Phase 5: Model Validity Assessment | ||
|
|
||
| Assess whether the simulation models are trustworthy. | ||
|
|
||
| 1. **Manufacturer models vs. generic**: Are critical components | ||
| (regulators, MOSFETs, op-amps) using manufacturer-provided SPICE | ||
| models? Generic models may not capture dropout behavior, thermal | ||
| shutdown, or current limiting accurately. | ||
|
|
||
| 2. **Model completeness**: Do the models include the behaviors that | ||
| matter for this simulation? | ||
| - Regulator models: dropout, PSRR, load transient response, | ||
| current limit | ||
| - MOSFET models: gate threshold variation, RDS(on) vs. temperature | ||
| - Capacitor models: ESR, DC bias derating, temperature coefficient | ||
|
|
||
| 3. **Parasitic accuracy**: Are PCB parasitics (trace resistance, | ||
| via inductance, ground impedance) included where they affect | ||
| results? Flag if the simulation assumes ideal connections for | ||
| high-current or high-frequency paths. | ||
|
|
||
| 4. **Confidence classification**: For each simulation result, assess | ||
| model confidence: | ||
| - **High**: manufacturer model, parasitics included, matches | ||
| datasheet typical curves | ||
| - **Medium**: manufacturer model but missing some parasitic effects, | ||
| or generic model for a non-critical component | ||
| - **Low**: generic model for a critical component, or significant | ||
| parasitics omitted | ||
|
|
||
| ## Phase 6: Findings Summary | ||
|
|
||
| Compile findings from all phases. | ||
|
|
||
| 1. **For each finding**, document: | ||
| - Phase that discovered it (use the exact phase label, e.g., | ||
| "Phase 1 — Simulation Setup Review", "Phase 4 — Corner-Case | ||
| and Worst-Case Assessment") | ||
| - Affected component(s) and net(s) | ||
| - The specification constraint involved (if applicable) | ||
| - Severity (Critical / High / Medium / Low / Informational) | ||
| - Remediation: simulation fix (re-run with correct setup) vs. | ||
| design fix (circuit change needed) vs. coverage gap (additional | ||
| simulation needed) | ||
|
|
||
| 2. **Produce a simulation coverage summary**: | ||
| - Constraints checked / total specification constraints | ||
| - Corners simulated / total relevant corners | ||
| - Model confidence: count of High / Medium / Low confidence results | ||
| - Overall simulation health assessment | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,139 @@ | ||
| <!-- SPDX-License-Identifier: MIT --> | ||
| <!-- Copyright (c) PromptKit Contributors --> | ||
|
|
||
| --- | ||
| name: validate-simulation | ||
| description: > | ||
| Review circuit simulation output (SPICE, power budget, thermal | ||
| analysis) against specification constraints. Verifies simulation | ||
| setup, extracts results, checks constraint compliance, assesses | ||
| corner-case coverage, and evaluates model validity. | ||
| persona: electrical-engineer | ||
| protocols: | ||
| - guardrails/anti-hallucination | ||
| - guardrails/self-verification | ||
| - analysis/simulation-validation | ||
| format: investigation-report | ||
| params: | ||
| project_name: "Name of the project or board being validated" | ||
| simulation_output: "The simulation results to review — SPICE output, waveform data, power budget table, thermal analysis report" | ||
| simulation_setup: "Simulation configuration — schematic/netlist used, stimulus conditions, component models, measurement points" | ||
| requirements_doc: "Hardware requirements document with quantitative constraints to validate against" | ||
| datasheets: "Relevant component datasheets — especially regulator characteristics, thermal resistance, and operating limits" | ||
| context: "Additional context — target operating environment, known design margins, areas of concern" | ||
| audience: "Who will read the output — e.g., 'HW engineer before PCB order', 'design review board', 'test engineer planning validation'" | ||
| input_contract: null | ||
| output_contract: | ||
| type: investigation-report | ||
| description: > | ||
| A simulation review report with findings covering setup issues, | ||
| constraint violations, corner-case gaps, and model validity | ||
| concerns. Includes simulation coverage summary. | ||
| --- | ||
|
|
||
| # Task: Validate Simulation Results | ||
|
|
||
| You are tasked with performing a **systematic review** of circuit | ||
| simulation output against specification constraints. | ||
|
|
||
| ## Inputs | ||
|
|
||
| **Project Name**: {{project_name}} | ||
|
|
||
| **Simulation Output**: | ||
| {{simulation_output}} | ||
|
|
||
| **Simulation Setup**: | ||
| {{simulation_setup}} | ||
|
|
||
| **Requirements Document**: | ||
| {{requirements_doc}} | ||
|
|
||
| **Datasheet Excerpts**: | ||
| {{datasheets}} | ||
|
|
||
| **Context**: {{context}} | ||
|
|
||
| **Audience**: {{audience}} | ||
|
|
||
| ## Instructions | ||
|
|
||
| 1. **Apply the simulation-validation protocol** systematically. | ||
| Execute all six phases in order. Do not skip phases — document | ||
| phase coverage in the **Investigation Scope** section. | ||
|
|
||
| 2. **Phase 1 (Setup Review) catches the most impactful issues.** | ||
| A simulation with wrong stimulus conditions or missing models | ||
| produces meaningless results. Verify setup before interpreting | ||
| results. | ||
|
|
||
| 3. **Cross-reference results to requirements.** For each constraint | ||
| check in Phase 3, cite the REQ-ID from the requirements document. | ||
| For datasheet limits, cite the datasheet section. | ||
|
|
||
| 4. **Apply the anti-hallucination protocol** throughout: | ||
| - Only assess results that are present in the provided simulation | ||
| output — do NOT extrapolate to conditions that were not simulated | ||
| - Do NOT fabricate simulation results or component parameters | ||
| - If a component's SPICE model is not identified in the setup, | ||
| flag it as a limitation — do not assume model quality | ||
| - Distinguish between [KNOWN] (simulation output shows), | ||
| [INFERRED] (derived from simulation patterns), and [ASSUMPTION] | ||
| (depends on information not in the simulation) | ||
|
|
||
| 5. **Format the output** according to the investigation-report format: | ||
| - List all findings in a single **Findings** section ordered | ||
| strictly by severity (Critical first) | ||
| - For each finding, indicate the protocol phase under **Category** | ||
| using phase number and title (e.g., "Phase 1: Simulation Setup | ||
| Review", "Phase 4: Corner-Case and Worst-Case Assessment") | ||
| - Under **Location**, identify the specific component, net, or | ||
| measurement point | ||
| - Under **Evidence**, include the specific simulation values and | ||
| the constraint they violate or the gap they expose | ||
|
|
||
| 6. **Prioritize findings** by impact on design confidence: | ||
| - **Critical**: Simulation shows constraint violation under nominal | ||
| conditions — design change likely needed | ||
| - **High**: Simulation shows marginal compliance or constraint | ||
| violation under corner conditions — design risk | ||
| - **Medium**: Simulation setup issue that may invalidate results — | ||
| re-simulation needed before drawing conclusions | ||
| - **Low**: Missing corner case or model concern that doesn't | ||
| threaten compliance under simulated conditions | ||
| - **Informational**: Observation about simulation quality or | ||
| suggestion for additional analysis | ||
|
|
||
| 7. **Apply the self-verification protocol** before finalizing: | ||
| - Randomly re-check at least 3 constraint compliance assessments | ||
| against the provided simulation values | ||
| - Verify the corner-case coverage assessment is complete | ||
| - Confirm every phase is documented in the coverage summary | ||
|
|
||
| ## Non-Goals | ||
|
|
||
| - Do NOT run simulations — this is a review of existing simulation | ||
| output, not a simulation execution task | ||
| - Do NOT redesign the circuit — report findings with remediation | ||
| suggestions (re-simulate vs. design change) | ||
| - Do NOT review the schematic itself — this validates simulation | ||
| results, not schematic correctness (use `review-schematic` for that) | ||
| - Do NOT perform quantitative budget validation — this reviews | ||
| simulation results for setup validity and coverage. For formal | ||
| margin analysis of budget artifacts, use `validate-budget` | ||
|
|
||
| ## Quality Checklist | ||
|
|
||
| Before finalizing, verify: | ||
|
|
||
| - [ ] All 6 protocol phases were executed and documented | ||
| - [ ] Simulation setup was verified before interpreting results | ||
| - [ ] Every specification constraint was mapped to a simulation result | ||
| or flagged as a coverage gap | ||
| - [ ] Input voltage, load, and temperature corners were assessed | ||
| - [ ] Component model quality was evaluated for critical components | ||
| - [ ] Every finding cites specific simulation values and components | ||
| - [ ] Every finding has a severity and remediation type (re-simulate | ||
| vs. design change vs. add coverage) | ||
| - [ ] Simulation coverage summary is complete | ||
| - [ ] No fabricated simulation results |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.