[workshop-sim] Workshop Simulation Report — 2026-07-16 (Run #49, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-07-16
• Students simulated: 46 × 1 000 Monte Carlo runs (49 128 cumulative runs per student)
• Workshop steps available: 15 main logical steps / 53 step files
• Overall completion rate: ~0% — no student completed all steps in any run
• Highest-dropout step: 08-run-workflow (64.9% dropout rate — auth barrier)
• Lowest curriculum quality step: 05-agentic-workflows-intro.md (overall 7.93/10, cognitive_load 4.4)
• Learning KPI index: 8.22/10 (active_learning 5.18 · checkpoint_quality 10.00 · scaffolding 9.91)

---

Critical Findings

1. Step 8 is the single biggest blocker (64.9% dropout; access barrier). Auth demand = 1.0 because model access requires either copilot-requests: write permission or a third-party API secret (ANTHROPIC_API_KEY / OPENAI_API_KEY). Students who reach this step without completing the model-access gate fail in the vast majority of runs. The step itself is well-written and UI-first, so the failure is not instructional — it is an access prerequisite that must be resolved before clicking Run.

2. Step 2 (environment setup) hits beginners and enterprise users hardest (12.2% dropout). No-coding background students (6 profiles) fail 30–69% of individual runs at setup, compounding with later-step failures to keep their cumulative completion near zero. Enterprise/GHES users face additional proxy/auth friction the Codespace path cannot absorb alone.

3. 05-agentic-workflows-intro.md causes 8.2% dropout and scores lowest overall (7.93/10). It introduces 25 new concepts in 1 417 words, producing high concept density for beginners and enterprise-dev profiles who already struggle with domain vocabulary. The low cognitive_load score (4.4/10) reflects this density.

4. Learning quality health is strong for students who persist. The learning KPI index of 8.22/10 — with checkpoint_quality at 10.00 and scaffolding at 9.91 — confirms that the curriculum's hands-on activities, checkpoints, and scaffolding are excellent. Active learning (5.18) is the only dimension below 6.0 and is the primary area for improvement without lowering the cognitive bar.

---

Top Repairs to Prioritize

Some dropout is expected and acceptable. Repairs below target access barriers and active-learning gaps. None lower the cognitive demand or remove practice.

1. Add a pre-flight model-access check before Step 8 — gate the run button behind a 3-item checklist verifying engine selection, permission block, and secret presence. (completion impact: ↑↑ · learning KPI impact: ↔)

2. Reduce concept density in 05-agentic-workflows-intro.md by splitting into two sections: conceptual (for beginners) and practitioner fast-path (for actions-users and advanced). Keeps the skip condition but adds a lightweight progressive disclosure structure. (completion impact: ↑ · learning KPI impact: ↑ — improves active_learning score)

3. Add an explicit enterprise / proxy setup callout to Step 2 (setup) with a link to the enterprise side quest placed earlier in the flow, before the terminal verification step. (completion impact: ↑ · learning KPI impact: ↔)

---

Dropout by step

Step	Dropout rate	Total failures (46×1 000)	Failure mode	Top reason
08-run-workflow	64.9%	29 834	Access barrier	Missing copilot-requests: write or API key secret
02-setup	12.2%	5 593	Access barrier	Auth + terminal complexity for beginners; enterprise/proxy friction
05-agentic-intro	8.2%	3 749	Learning barrier	25 new concepts, high concept density (4.4/10 cognitive_load)
04-actions-intro	4.8%	2 194	Learning barrier	YAML/CI concepts unfamiliar to no-coding backgrounds
07-first-workflow	4.7%	2 171	Access barrier	Terminal + auth demand; ui_preferred users hit CLI-only gh aw compile
03-create-your-repo	3.7%	1 686	Access barrier	gh auth not configured; impatient students skip prerequisite check
06-install-gh-aw	1.2%	551	Access barrier	Extension install fails silently in some terminal environments
13-schedule	0.3%	131	Learning barrier	Enterprise students encounter CRON/schedule concepts without Actions background

Learning quality KPIs

Step file	Overall	active_learning	checkpoint_quality	scaffolding	KPI index	Repair priority
05-agentic-workflows-intro.md	7.93	6.0	10	10.0	8.55	High — lowest overall; concept density
10-choose-your-scenario.md	8.04	3.0	10	10.0	7.45	Medium — low active_learning
14-next-steps.md	8.10	1.8	10	10.0	7.02	Medium — low active_learning (closing step)
07-your-first-workflow.md	8.64	3.2	10	10.0	7.53	High — 4.7% dropout + low active_learning
08-run-your-workflow.md	8.96	4.9	10	10.0	8.15	High — 64.9% dropout (access, not learning)
11c-build-pr-reviewer.md	8.47	6.1	10	5.0	7.22	Medium — only step with scaffolding < 10
12-test-and-iterate.md	8.48	3.0	10	10.0	7.45	Medium — low active_learning
02a-setup-codespace.md	8.21	4.7	10	10.0	8.07	High — 12.2% dropout
Cohort mean	8.79	5.18	10.00	9.91	8.22	—

Curriculum quality metrics

Step file	Overall	Lowest rubric dimension	Recommended repair focus
05-agentic-workflows-intro.md	7.93	cognitive_load (4.4)	Split concept-heavy intro; add practitioner fast-path
10-choose-your-scenario.md	8.04	active_learning (3.0)	Add hands-on decision activity before scenario selection
14-next-steps.md	8.10	active_learning (1.8)	Add a closing reflection activity
16-connect-data-source.md	8.11	active_learning (5.1)	Add a guided practice task
02a-setup-codespace.md	8.21	active_learning (4.7)	Add a terminal verification exercise post-setup
11c-build-pr-reviewer.md	8.47	scaffolding (5.0)	Add intermediate checkpoints or build-up steps
12-test-and-iterate.md	8.48	active_learning (3.0)	Add structured iteration exercise
11a-build-daily-status-terminal.md	8.46	active_learning (5.8)	Minor: add a self-test step
10b-design-daily-docs.md	8.60	active_learning (3.2)	Add a design rationale prompt
10c-design-pr-reviewer.md	8.58	active_learning (2.9)	Lowest active_learning outside of closing steps

Segment breakdowns

By technical level (all show 0% completion due to Step 8 auth wall, but relative failure distribution differs)

Level	n	Most common failure step	Avg failures/student
beginner	11	02-setup (no-coding) / 08-run-workflow (web-dev)	High — auth + setup
github-basic	14	08-run-workflow	Very high — auth barrier
actions-user	14	08-run-workflow	Very high — auth barrier
advanced	7	08-run-workflow	High — auth barrier only

By personality

Personality	n	Key pattern
curious	10	Progresses well until auth wall
methodical	11	Reads carefully; hits auth wall later but more consistently
impatient	8	High setup dropout (skips prerequisite checks); fast auth failures
confused	9	High setup dropout; auth wall compounds confusion
skeptical	8	Skips conceptual steps; auth wall hits hard

By UI preference

UI preferred	n	Pattern
true (18)	18	Setup easier; auth wall equally severe; no UI alternative for gh aw compile
false (28)	28	Terminal-comfortable; auth wall identical

Notable student journeys (3)

Surprising success — Jamie Liu (Student #2, beginner/no-coding/methodical/ui=True)
Jamie spread failures more evenly (setup 258, actions-intro 219, run-workflow 173) rather than being blocked entirely at one step. The methodical personality means Jamie reads every prerequisite carefully, which partially mitigates the setup confusion, but the auth barrier at Step 8 still prevents any full completion. If Step 8's pre-flight gate were clearer, Jamie's profile is the most likely to complete among beginners.

Unexpected dropout — Hayden Brooks (Student #40, github-basic/program-manager/02-setup: 1000/1000)
Hayden fails 100% of runs at setup, despite being a github-basic user. A program manager background means limited terminal tolerance; the setup step's dual-path structure (Codespace vs local) likely triggers decision paralysis. Despite good troubleshootingSupport (0.96), the complexity (0.95) of the combined setup step exceeds Hayden's profile tolerance in every run.

Content-gap case — Sam Torres (Student #39, beginner/enterprise-dev/05-agentic-intro: 521 failures)
Sam is a beginner with an enterprise-dev background — meaning they understand software concepts but have no GitHub Actions experience. The 05-agentic-workflows-intro.md step, with its 25 new concepts, hits Sam before the Actions foundation is solid enough. The high workflowCompileCueCount (10) in that step implies the concept of compiling workflow Markdown is introduced here — a step too early for this profile. Sam then re-fails at 13-schedule (131 failures), suggesting cron/schedule concepts also lack sufficient enterprise-context scaffolding.

Generated by 🔬 Workshop Student Simulator · 79.9 AIC · ⌖ 6.75 AIC · ⊞ 8.9K · ◷

• expires on Jul 17, 2026, 6:27 PM UTC