Markdown Editor

AI Feature PRD Template

PRD for an AI-powered feature: why AI, failure and uncertainty UX, metrics, eval thresholds, cost per request, rollout

AI & Dữ liệuĐầy đủfeatureprdmath
Mở trong editorTải .md

Cách dùng: Use it when a product feature depends on an ML model or LLM. Agree the problem, evidence and why AI first, and set acceptance thresholds before any evaluation run. The main pitfall is designing only the happy path and not what users see when the model is wrong or uncertain.

Xem trước

AI Feature PRD: [Feature name]

Product requirements document for a feature powered by a machine learning model or large language model. Beyond a normal PRD it covers why AI is the right approach, how the experience handles wrong or uncertain outputs, model options, data needs, product and model metrics, an evaluation plan with acceptance thresholds, responsible AI review, human oversight, cost per request, latency, experimentation, monitoring and legal review. Based on common industry practice. Replace every [bracketed] value and every row marked Example.

Document control

FieldValue
Document ID[PRD-AI-001]
Feature[Feature name]
Version[0.1]
Status[Draft / In review / Approved for build / Approved for launch]
Product owner[@pm]
Engineering lead[@eng-lead]
ML / AI lead[@ml-lead]
Design lead[@designer]
Last updatedYYYY-MM-DD
Target launchYYYY-MM-DD

Approvals

RoleNameGateDecisionDate
Product owner[@name]Build[Approve / Reject]YYYY-MM-DD
Engineering lead[@name]Build[Decision]YYYY-MM-DD
Responsible AI reviewer[@name]Launch[Decision]YYYY-MM-DD
Privacy / legal[@name]Launch[Decision]YYYY-MM-DD
Security[@name]Launch[Decision]YYYY-MM-DD

Revision history

VersionDateAuthorChange summary
0.1YYYY-MM-DD[@name]First draft
0.2YYYY-MM-DD[@name]Example: Added confidence display after usability test
1.0YYYY-MM-DD[@name]Example: Approved for build

Contents

1. Problem and users

1.1 Problem statement

[Who has what problem, when, and what it costs them.] Example: Small business owners using our invoicing app spend a median of 11 minutes per week categorising expenses by hand; 34% of expenses stay uncategorised at month end, which breaks their tax reports.

1.2 Evidence

SourceFinding
Example: Usage analytics (90 days)2.1 million expenses created; 34% uncategorised after 30 days
Example: 12 user interviews9 of 12 named categorisation as the most tedious task
Example: Support tickets6% of tickets about wrong tax totals trace back to missing categories

1.3 Users and personas

PersonaContextNeed
Example: Sole traderUploads receipts from phoneFast, mostly correct categories without thinking
Example: BookkeeperManages 20 client accountsBulk review, control and an audit trail

2. Goals and non-goals

GoalsNon-goals
Example: Cut uncategorised expenses at month end from 34% to under 10%Giving tax advice
Example: Keep users in control: every suggestion is visible and reversibleAuto-filing tax returns
Example: Work on mobile and webSupporting scanned paper ledgers in v1

3. Why AI

ApproachCoverageAccuracyEffortVerdict
Example: Rules on merchant name48% of expenses97% on coveredLowKeep as first layer
Example: User-defined rules15% moreHighUsers rarely create rulesKeep as override
Example: Classification model on merchant, amount, memo95%88% top-1 estimatedMediumSelected for the rest

[Summarise why AI is needed and what happens if it underperforms.] Example: Rules cover half of expenses; the long tail of merchants needs a learned model. If accuracy falls below threshold, the feature falls back to rules only.

4. User experience

4.1 Main flow

  1. User adds an expense (photo, bank import or manual).
  2. System suggests a category within [1] second and marks it as suggested.
  3. User accepts with one tap, picks another category, or ignores.
  4. Corrections are stored and used to personalise future suggestions.

4.2 Uncertainty display

ConfidenceDisplayBehaviour
Example: >= 0.80Category with "Suggested" labelOne-tap accept
Example: 0.50 to 0.79Top 3 categories as chipsUser must choose
Example: < 0.50"Choose a category"No suggestion shown

4.3 Failure and edge states

StateWhat the user seesNotes
Wrong suggestion acceptedEdit at any time; change history shows "suggested by AI"Bulk undo for bookkeepers
Model unavailableRules still work; others show "Choose a category"No error dialog
Slow responseSuggestion appears when ready; never blocks savingTimeout [2] s
Sensitive category (for example medical)No auto-suggestionPrivacy rule
New user, no historyGeneric model onlyPersonalisation after [20] corrections

4.4 Transparency and control

[Example: An info icon explains "Categories are suggested by an automated model based on merchant and amount. Check them before filing." Settings let users turn suggestions off.]

5. Requirements

IDRequirementPriorityVerification
FR-01The system shall suggest a category for each new expense not matched by a ruleMustFunctional test
FR-02The system shall show suggestions as visibly distinct from user-set valuesMustUI review
FR-03Users shall be able to accept, change or dismiss a suggestion in one actionMustUsability test
FR-04Bookkeepers shall be able to review all AI-set categories in a filtered listShouldFunctional test
NFR-01p95 suggestion latency shall be at most 800 msMustLoad test
NFR-02Top-1 accuracy on the held-out set shall be at least 0.88MustOffline evaluation
NFR-03No customer data shall be used to train shared models without opt-inMustPrivacy review

6. Model and approach options

OptionQuality (est.)Cost / 1k requestsp95 latencyPrivacyVerdict
Example: Gradient-boosted classifier, self-hosted0.86 top-1$0.0240 msData stays in regionFallback
Example: Fine-tuned small transformer, self-hosted0.90$0.10120 msData stays in regionSelected
Example: Hosted LLM, zero-shot with category list0.84$1.801,400 msData sent to vendorRejected: cost and latency

Decision: [Example: fine-tuned small transformer. Revisit if top-1 drops below 0.86 or if multilingual support is required.]

7. Data needs

DataPurposeSourceVolumeLegal basis / consentIssues
Example: Historical categorised expensesTrainingProduct database3.4 M rowsContract + opt-in for shared modelLabel noise from bulk edits
Example: Held-out set, re-labelledEvaluationSample of 5,0005,000SameTwo annotators, kappa >= 0.8
Example: Merchant, amount, memoRuntime featuresLive expensePer requestContractMemos may contain personal data

See [Datasheet for the training dataset] for full documentation.

8. Success metrics and evaluation

8.1 Metrics

IDMetricTypeBaselineTarget
M-01Example: Uncategorised expenses at month endProduct34%< 10%
M-02Example: Suggestion acceptance rateProductn/a>= 75%
M-03Example: Top-1 accuracy (held-out)Modeln/a>= 0.88
M-04Example: Accuracy gap between smallest and largest business segmentFairnessn/a<= 5 points
G-01Example: Support tickets about wrong categoriesGuardrail6%Not higher

8.2 Evaluation plan and acceptance thresholds

StageMethodPass thresholdOwner
OfflineHeld-out set, stratified by segment and languageM-03 and M-04 met@ml-lead
DogfoodInternal accounts for 2 weeksNo severity-1 issues@pm
A/B test10% of new expensesM-02 >= 75%; G-01 not worse at 95% confidence@data-scientist

The minimum sample per arm to detect a change of δ\delta in a proportion pp with significance α\alpha and power 1−β1-\beta is approximately

n≈2 (z1−α/2+z1−β)2 p(1−p)δ2n \approx \frac{2\,(z_{1-\alpha/2} + z_{1-\beta})^{2}\, p(1-p)}{\delta^{2}}

Example: p=0.06p = 0.06, δ=0.01\delta = 0.01, α=0.05\alpha = 0.05, 1−β=0.81-\beta = 0.8 gives n≈2×7.85×0.0564/0.0001≈8,860n \approx 2 \times 7.85 \times 0.0564 / 0.0001 \approx 8{,}860 per arm.

9. Responsible AI and human oversight

TopicAssessmentMitigation
Harm from errorsExample: Wrong categories can lead to incorrect tax filingsSuggestions labelled; review list; never auto-file
FairnessExample: Lower accuracy for non-English memosSegment metrics; threshold M-04
TransparencyExample: Users must know a suggestion is automatedLabel and info text (4.4)
Over-relianceExample: Users accept without checkingMonthly "review AI categories" prompt
Human oversightExample: Users and bookkeepers can override every suggestion; disable switch per accountSettings toggle
Regulatory classificationExample: Not a high-risk use under the EU AI Act per legal reviewRe-check if scope changes

10. Cost and latency

For a self-hosted model, cost per request is driven by GPU time. With instance cost chc_h per hour and sustained throughput rr requests per second per instance at target utilisation uu:

Creq=ch3600⋅r⋅uC_{\text{req}} = \frac{c_h}{3600 \cdot r \cdot u}

Example: ch=$1.20c_h = \text{\textdollar}1.20, r=150r = 150, u=0.6u = 0.6 gives Creq≈$0.0000037C_{\text{req}} \approx \text{\textdollar}0.0000037, about $0.004\text{\textdollar}0.004 per thousand requests; at 2.5 million requests per month, about $9\text{\textdollar}9 plus a fixed minimum of two instances (≈$1,730\approx \text{\textdollar}1{,}730 per month).

Stepp95 budget
Rule engine20 ms
Feature lookup60 ms
Model inference150 ms
Network and rendering300 ms
Total<= 800 ms

11. Rollout and experimentation

StageAudienceDurationGo criteria
InternalStaff accounts2 weeksNo severity-1 bugs
A/B10% of new expenses3 weeksThresholds in 8.2
Ramp50%, then 100%2 weeksGuardrails stable

Rollback: [Example: feature flag expense_ai_suggest off returns to rules only within one minute; stored suggestions stay labelled].

12. Monitoring

SignalThresholdActionOwner
Acceptance rate (7-day)Example: < 70%Investigate drift@ml-lead
Input drift (merchant mix)Example: PSI > 0.2Retrain review@ml-lead
p95 latency> 800 ms for 30 minPage on-call@eng-lead
Segment accuracy gap> 5 pointsFairness review@responsible-ai
Cost per month> 120% of budgetReview capacity@eng-lead

13. Legal, privacy and security

ReviewOutcomeLink
Privacy (DPIA screening)Example: DPIA not required; memos minimised before inference[Link]
Data use termsExample: Opt-in for shared training added to settings[Link]
SecurityExample: Model endpoint internal only; inputs validated[Link]
Vendor reviewExample: Not applicable (self-hosted)[Link]
User-facing disclosuresExample: Help article and in-product label approved[Link]

14. Launch checklist

ItemEvidenceDone
Offline thresholds met[Evaluation report][ ]
Failure states implemented and tested[Test run][ ]
A/B results reviewed[Analysis][ ]
Responsible AI review signed off[Link][ ]
Privacy and legal sign-off[Link][ ]
Monitoring dashboards and alerts live[Dashboard][ ]
Support team briefed; help article published[Link][ ]
Rollback tested[Record][ ]

15. Open questions and appendix

IDQuestionOwnerDue
Q-01[Should bookkeepers be able to train client-specific models?][@name]YYYY-MM-DD
Q-02[Which languages for v2?][@name]YYYY-MM-DD
TermMeaning
Top-1 accuracyShare of cases where the highest-scored category is correct
Acceptance rateShare of shown suggestions the user keeps
PSIPopulation stability index, a measure of distribution shift
Guardrail metricA metric that must not get worse when the feature launches

Standard and licence note

Based on common industry practice for product requirements of AI-powered features, including standard sample-size estimation for A/B tests. Original text written for this collection; example numbers and prices are illustrative. Template licence: MIT.

Template liên quan