AIプロダクトマネージャーキット

AI problem framing: is AI needed, value hypothesis, data availability, feasibility, risk tier preview, success metrics, non-AI alternatives, go/no-go

Screen a proposed AI use case: data, affected people, AI Act prohibited/high-risk/transparency questions, tier, reviews

Copilot and agent requirements: jobs, autonomy levels, actions and undo, approval points, eval acceptance criteria, latency and cost, failure UX, rollout

PRD for an AI-powered feature: why AI, failure and uncertainty UX, metrics, eval thresholds, cost per request, rollout

One metric or KPI: business question, plain definition, formula, grain, SQL sketch, filters, targets, caveats

Design review on the 18 Guidelines for Human-AI Interaction: scenarios, evidence log, ratings and severity per guideline, scorecard, trade-offs, findings

Task and rubric, rater recruitment and training, ethics and consent, sample size, blinding, Cohen's/Fleiss' kappa, Krippendorff's alpha, QC, analysis

LLM app evaluation: test sets and red team, metrics, LLM-as-judge calibration, human review, CIs, release gates

Cost per request and task, tokens in and out, caching and routing scenarios, self-hosted GPU vs API break-even, utilisation, sensitivity table, cost guardrails

Launch scope and tier, 18 AI gates with owners, eval results vs thresholds, red team findings, Article 50 and register checks, exceptions, go/no-go, post-launch

System-level card for a deployed AI product: components, uses, limitations, safety and bias evals, red-teaming

Article 50 role and scope, interaction disclosure, C2PA and watermark marking, robustness tests, labels, exceptions, Code of Practice mapping, China labelling

Post-market monitoring plan for a high-risk AI system: data collection, thresholds, reviews, Article 73 incident reporting

LLM and agent postmortem: impact, harmful output, data exposure, timeline, trace replay, eval regression, AI failure taxonomy, actions, notification check