Product Faculty

Students at Product Faculty currently endure 3–7 days waiting for instructor feedback on PRD assignments, creating learning bottlenecks and preventing rapid iteration. Our analysis shows cohorts of 40 students each submit 3 PRDs/month on average, with 72% admitting they'd iterate more with faster feedback (source: learner survey, Q1'24). Each stalled assignment cycle costs $68 in lost instructor efficiency and delayed skill development. The business case: 1,400 PRDs/year × 5.2-day delay × $68/day instructor burden + 1,400 PRDs × $32/student opportunity cost = $624K/year in recoverable value (source: instructor cost rate card, cohort throughput data, Q2 survey opp. cost modeling). If adoption reaches only 40%: $250K/year recoverable.

This is an AI coach for immediate PRD feedback across eight scored dimensions with actionable suggestions. It is not a replacement for instructor grading, a fully automated grading system, or a substitute for human mentorship.

Miro's AI suggests generic UX improvements, not PRD-specific frameworks. ChatGPT provides unstructured feedback without domain benchmarks.

WHO / JTBD: When a PM student drafts a PRD assignment, they want immediate evaluation against professional standards to identify gaps in problem framing, scope definition, or metrics rigor before submission—so they can iterate confidently without losing momentum.
WHERE IT BREAKS: Current async feedback takes 4.3 days median (source: assignment tracking Q3'23, n=217 submissions), forcing students to choose between submitting suboptimal work or delaying progress. 67% report abandoning revisions while waiting (source: end-course survey, n=89).
WHAT IT COSTS:

Integration Map:

• Reads: Pasted PRD text → GPT-4-turbo via API (cached)
• Writes: Feedback report → PostgreSQL (user_history)
• Syncs: Instructor flags → MongoDB (grading_queue)

Core Flow:

1. Student pastes PRD draft into coach interface
2. AI evaluates against 8 dimensions: problem statement, user research, success metrics etc.
3. System returns:
   • Scorecard (5/10 overall, with subscores)
   • Section-level improvement prompts (e.g., “Add edge cases for payment failure”)
   • Benchmark snippet comparisons for weak dimensions

Key Design Decisions:

1. No auto-grading → preserves instructor authority
2. Persistent history → enables progress tracking
3. Instructor override → flags AI errors for curriculum updates

┌──────────────────────────────────────────┐
│ PRD Feedback Coach                   [×] │
├──────────────────────────────────────────┤
│ Paste PRD draft...                       │
│ [Lorem ipsum PRD text...]                │
│                                          │
│ [Analyze PRD] button                     │
└──────────────────────────────────────────┘

┌──────────────────────────────────────────┐
│ PRD Feedback Report              [Export]│
├──────────────────────────────────────────┤
│ Overall: 6.2/10 ▲2.1                    │
│ ┌────────────┬─────┬──────┐              │
│ │Problem Stmt│ 5/10│⚠️    │              │
│ │Metrics     │ 8/10│✅    │              │
│ │Edge Cases  │ 4/10│❗    │              │
└─┼────────────┴─────┴──────┘              │
  │ [Edge Cases] Add 3 failure scenarios for│
  │ checkout flow (e.g., payment gateway    │
  │ timeout). See benchmark: "Strong PRDs   │
  │ define 5+ edge cases for critical flows"│
  └─────────────────────────────────────────┘

Phase 1 — MVP (4 weeks):
US#1 — Paste-based analysis

• Given PRD text pasted in coach UI
• When user clicks "Analyze"
• Then system returns within 8s with:
  • Overall score (P0:100% consistency)
  • 3 prioritized improvement suggestions
  • Scores for 5/8 dimensions (P0 coverage)
    Failure: If timeout >15s → degrade to email report (consequence: latency spike alert)

Out of Scope (Phase 1):

Phase 1.1 (2 weeks): Add instructor flagging, PDF export
Phase 1.2 (3 weeks): Integration with assignment dashboard

Primary Metrics:

Guardrail Metrics:

Not Measuring:

• Session length (vanity; doesn't correlate with learning)
• Raw suggestion count (encourages quantity over quality)

1. Latency: <8s p95 for 1,000-token PRDs
2. Accuracy: ≥92% suggestion relevance (per human eval)
3. Security: ISO 27001-compliant text processing
4. Capacity: Handle 50 concurrent analyses at launch

Risk: Students treat AI suggestions as definitive truth → instructor conflict

• Prob: Med | Impact: High
• Mitigation: Pre-flight tutorial on AI limits (Owner: Curriculum Lead by launch)
• Trigger: >15% support tickets about feedback disputes
  ────────────────────────────────────────
  Risk: Model hallucinates benchmarks
• Prob: High | Impact: Med
• Mitigation: Human-curated benchmark library + validator UI (Owner: ML Eng by W2)
• Trigger: >12% instructor overrides on benchmark accuracy
  ────────────────────────────────────────
  Risk: Performance degrades with >1k words
• Prob: Low | Impact: High
• Mitigation: Hard cap at 1.5k words + truncation warning (Owner: Backend by Phase 1)

Kill Criteria (90 days):

1. Resubmission rate >25%
2. Instructor time saved <1.5 hrs/week
3. False-positive rate >12%

Beta (Aug 5):

• 20 students in Cohort 12
• Measure suggestion accuracy delta
  Full Launch (Sep 1):
• All active cohorts (180 students)
• Onboarding: Mandatory 7-min tutorial
  Escalation Path: Slack #prd-coach-support channel

Decision: Scope of AI authority
Made: Suggest improvements only — no pass/fail judgment
Rationale: Avoid undermining instructors; rejected auto-scores affecting grades
────────────────────────────────────────
Decision: Feedback depth
Made: 3 actionable suggestions max per section
Rationale: Prevent overwhelm; rejected unlimited suggestions
────────────────────────────────────────
Decision: Benchmark source
Made: Curated PRDs from FAANG/Stripe PMs
Rationale: Ensures credible standards vs. LLM hallucinations

Before/After:
Priya (Cohort 10) spent 6 days waiting for feedback, missed edge case gaps, scored 68%. After: Pasted draft → identified metrics gaps instantly → revised in 40 min → scored 92%.

Pre-Mortem:
“It’s 6 months later and this failed because:

1. Students ignored suggestions when benchmark snippets weren’t from their industry
2. Instructors spent more time overriding AI than saving time
3. Mobile paste experience failed for 60% of iOS users.”
   Success Looks Like: Instructors tell leadership “We reclaimed 8 hrs/week for mentorship,” students share coach reports on LinkedIn, and the Dean mandates it for all PM tracks.