AI Feature PRD: [Feature name]
Product requirements document for a feature powered by a machine
learning model or large language model. Beyond a normal PRD it covers
why AI is the right approach, how the experience handles wrong or
uncertain outputs, model options, data needs, product and model metrics,
an evaluation plan with acceptance thresholds, responsible AI review,
human oversight, cost per request, latency, experimentation, monitoring
and legal review. Based on common industry practice. Replace every
[bracketed] value and every row marked Example.
Document control
| Field | Value |
|---|
| Document ID | [PRD-AI-001] |
| Feature | [Feature name] |
| Version | [0.1] |
| Status | [Draft / In review / Approved for build / Approved for launch] |
| Product owner | [@pm] |
| Engineering lead | [@eng-lead] |
| ML / AI lead | [@ml-lead] |
| Design lead | [@designer] |
| Last updated | YYYY-MM-DD |
| Target launch | YYYY-MM-DD |
Approvals
| Role | Name | Gate | Decision | Date |
|---|
| Product owner | [@name] | Build | [Approve / Reject] | YYYY-MM-DD |
| Engineering lead | [@name] | Build | [Decision] | YYYY-MM-DD |
| Responsible AI reviewer | [@name] | Launch | [Decision] | YYYY-MM-DD |
| Privacy / legal | [@name] | Launch | [Decision] | YYYY-MM-DD |
| Security | [@name] | Launch | [Decision] | YYYY-MM-DD |
Revision history
| Version | Date | Author | Change summary |
|---|
| 0.1 | YYYY-MM-DD | [@name] | First draft |
| 0.2 | YYYY-MM-DD | [@name] | Example: Added confidence display after usability test |
| 1.0 | YYYY-MM-DD | [@name] | Example: Approved for build |
Contents
1. Problem and users
1.1 Problem statement
[Who has what problem, when, and what it costs them.] Example: Small
business owners using our invoicing app spend a median of 11 minutes per
week categorising expenses by hand; 34% of expenses stay uncategorised at
month end, which breaks their tax reports.
1.2 Evidence
| Source | Finding |
|---|
| Example: Usage analytics (90 days) | 2.1 million expenses created; 34% uncategorised after 30 days |
| Example: 12 user interviews | 9 of 12 named categorisation as the most tedious task |
| Example: Support tickets | 6% of tickets about wrong tax totals trace back to missing categories |
1.3 Users and personas
| Persona | Context | Need |
|---|
| Example: Sole trader | Uploads receipts from phone | Fast, mostly correct categories without thinking |
| Example: Bookkeeper | Manages 20 client accounts | Bulk review, control and an audit trail |
2. Goals and non-goals
| Goals | Non-goals |
|---|
| Example: Cut uncategorised expenses at month end from 34% to under 10% | Giving tax advice |
| Example: Keep users in control: every suggestion is visible and reversible | Auto-filing tax returns |
| Example: Work on mobile and web | Supporting scanned paper ledgers in v1 |
3. Why AI
| Approach | Coverage | Accuracy | Effort | Verdict |
|---|
| Example: Rules on merchant name | 48% of expenses | 97% on covered | Low | Keep as first layer |
| Example: User-defined rules | 15% more | High | Users rarely create rules | Keep as override |
| Example: Classification model on merchant, amount, memo | 95% | 88% top-1 estimated | Medium | Selected for the rest |
[Summarise why AI is needed and what happens if it underperforms.]
Example: Rules cover half of expenses; the long tail of merchants needs a
learned model. If accuracy falls below threshold, the feature falls back to
rules only.
4. User experience
4.1 Main flow
- User adds an expense (photo, bank import or manual).
- System suggests a category within [1] second and marks it as suggested.
- User accepts with one tap, picks another category, or ignores.
- Corrections are stored and used to personalise future suggestions.
4.2 Uncertainty display
| Confidence | Display | Behaviour |
|---|
| Example: >= 0.80 | Category with "Suggested" label | One-tap accept |
| Example: 0.50 to 0.79 | Top 3 categories as chips | User must choose |
| Example: < 0.50 | "Choose a category" | No suggestion shown |
4.3 Failure and edge states
| State | What the user sees | Notes |
|---|
| Wrong suggestion accepted | Edit at any time; change history shows "suggested by AI" | Bulk undo for bookkeepers |
| Model unavailable | Rules still work; others show "Choose a category" | No error dialog |
| Slow response | Suggestion appears when ready; never blocks saving | Timeout [2] s |
| Sensitive category (for example medical) | No auto-suggestion | Privacy rule |
| New user, no history | Generic model only | Personalisation after [20] corrections |
4.4 Transparency and control
[Example: An info icon explains "Categories are suggested by an
automated model based on merchant and amount. Check them before filing."
Settings let users turn suggestions off.]
5. Requirements
| ID | Requirement | Priority | Verification |
|---|
| FR-01 | The system shall suggest a category for each new expense not matched by a rule | Must | Functional test |
| FR-02 | The system shall show suggestions as visibly distinct from user-set values | Must | UI review |
| FR-03 | Users shall be able to accept, change or dismiss a suggestion in one action | Must | Usability test |
| FR-04 | Bookkeepers shall be able to review all AI-set categories in a filtered list | Should | Functional test |
| NFR-01 | p95 suggestion latency shall be at most 800 ms | Must | Load test |
| NFR-02 | Top-1 accuracy on the held-out set shall be at least 0.88 | Must | Offline evaluation |
| NFR-03 | No customer data shall be used to train shared models without opt-in | Must | Privacy review |
6. Model and approach options
| Option | Quality (est.) | Cost / 1k requests | p95 latency | Privacy | Verdict |
|---|
| Example: Gradient-boosted classifier, self-hosted | 0.86 top-1 | $0.02 | 40 ms | Data stays in region | Fallback |
| Example: Fine-tuned small transformer, self-hosted | 0.90 | $0.10 | 120 ms | Data stays in region | Selected |
| Example: Hosted LLM, zero-shot with category list | 0.84 | $1.80 | 1,400 ms | Data sent to vendor | Rejected: cost and latency |
Decision: [Example: fine-tuned small transformer. Revisit if top-1 drops
below 0.86 or if multilingual support is required.]
7. Data needs
| Data | Purpose | Source | Volume | Legal basis / consent | Issues |
|---|
| Example: Historical categorised expenses | Training | Product database | 3.4 M rows | Contract + opt-in for shared model | Label noise from bulk edits |
| Example: Held-out set, re-labelled | Evaluation | Sample of 5,000 | 5,000 | Same | Two annotators, kappa >= 0.8 |
| Example: Merchant, amount, memo | Runtime features | Live expense | Per request | Contract | Memos may contain personal data |
See [Datasheet for the training dataset] for full documentation.
8. Success metrics and evaluation
8.1 Metrics
| ID | Metric | Type | Baseline | Target |
|---|
| M-01 | Example: Uncategorised expenses at month end | Product | 34% | < 10% |
| M-02 | Example: Suggestion acceptance rate | Product | n/a | >= 75% |
| M-03 | Example: Top-1 accuracy (held-out) | Model | n/a | >= 0.88 |
| M-04 | Example: Accuracy gap between smallest and largest business segment | Fairness | n/a | <= 5 points |
| G-01 | Example: Support tickets about wrong categories | Guardrail | 6% | Not higher |
8.2 Evaluation plan and acceptance thresholds
| Stage | Method | Pass threshold | Owner |
|---|
| Offline | Held-out set, stratified by segment and language | M-03 and M-04 met | @ml-lead |
| Dogfood | Internal accounts for 2 weeks | No severity-1 issues | @pm |
| A/B test | 10% of new expenses | M-02 >= 75%; G-01 not worse at 95% confidence | @data-scientist |
The minimum sample per arm to detect a change of δ in a proportion
p with significance α and power 1−β is approximately
n≈δ22(z1−α/2+z1−β)2p(1−p) Example: p=0.06, δ=0.01, α=0.05, 1−β=0.8
gives n≈2×7.85×0.0564/0.0001≈8,860 per arm.
9. Responsible AI and human oversight
| Topic | Assessment | Mitigation |
|---|
| Harm from errors | Example: Wrong categories can lead to incorrect tax filings | Suggestions labelled; review list; never auto-file |
| Fairness | Example: Lower accuracy for non-English memos | Segment metrics; threshold M-04 |
| Transparency | Example: Users must know a suggestion is automated | Label and info text (4.4) |
| Over-reliance | Example: Users accept without checking | Monthly "review AI categories" prompt |
| Human oversight | Example: Users and bookkeepers can override every suggestion; disable switch per account | Settings toggle |
| Regulatory classification | Example: Not a high-risk use under the EU AI Act per legal review | Re-check if scope changes |
10. Cost and latency
For a self-hosted model, cost per request is driven by GPU time. With
instance cost ch per hour and sustained throughput r requests per
second per instance at target utilisation u:
Creq=3600⋅r⋅uch Example: ch=$1.20, r=150, u=0.6 gives
Creq≈$0.0000037, about $0.004 per thousand
requests; at 2.5 million requests per month, about $9 plus a fixed
minimum of two instances (≈$1,730 per month).
| Step | p95 budget |
|---|
| Rule engine | 20 ms |
| Feature lookup | 60 ms |
| Model inference | 150 ms |
| Network and rendering | 300 ms |
| Total | <= 800 ms |
11. Rollout and experimentation
| Stage | Audience | Duration | Go criteria |
|---|
| Internal | Staff accounts | 2 weeks | No severity-1 bugs |
| A/B | 10% of new expenses | 3 weeks | Thresholds in 8.2 |
| Ramp | 50%, then 100% | 2 weeks | Guardrails stable |
Rollback: [Example: feature flag expense_ai_suggest off returns to
rules only within one minute; stored suggestions stay labelled].
12. Monitoring
| Signal | Threshold | Action | Owner |
|---|
| Acceptance rate (7-day) | Example: < 70% | Investigate drift | @ml-lead |
| Input drift (merchant mix) | Example: PSI > 0.2 | Retrain review | @ml-lead |
| p95 latency | > 800 ms for 30 min | Page on-call | @eng-lead |
| Segment accuracy gap | > 5 points | Fairness review | @responsible-ai |
| Cost per month | > 120% of budget | Review capacity | @eng-lead |
13. Legal, privacy and security
| Review | Outcome | Link |
|---|
| Privacy (DPIA screening) | Example: DPIA not required; memos minimised before inference | [Link] |
| Data use terms | Example: Opt-in for shared training added to settings | [Link] |
| Security | Example: Model endpoint internal only; inputs validated | [Link] |
| Vendor review | Example: Not applicable (self-hosted) | [Link] |
| User-facing disclosures | Example: Help article and in-product label approved | [Link] |
14. Launch checklist
| Item | Evidence | Done |
|---|
| Offline thresholds met | [Evaluation report] | [ ] |
| Failure states implemented and tested | [Test run] | [ ] |
| A/B results reviewed | [Analysis] | [ ] |
| Responsible AI review signed off | [Link] | [ ] |
| Privacy and legal sign-off | [Link] | [ ] |
| Monitoring dashboards and alerts live | [Dashboard] | [ ] |
| Support team briefed; help article published | [Link] | [ ] |
| Rollback tested | [Record] | [ ] |
15. Open questions and appendix
| ID | Question | Owner | Due |
|---|
| Q-01 | [Should bookkeepers be able to train client-specific models?] | [@name] | YYYY-MM-DD |
| Q-02 | [Which languages for v2?] | [@name] | YYYY-MM-DD |
| Term | Meaning |
|---|
| Top-1 accuracy | Share of cases where the highest-scored category is correct |
| Acceptance rate | Share of shown suggestions the user keeps |
| PSI | Population stability index, a measure of distribution shift |
| Guardrail metric | A metric that must not get worse when the feature launches |
Standard and licence note
Based on common industry practice for product requirements of AI-powered
features, including standard sample-size estimation for A/B tests.
Original text written for this collection; example numbers and prices are
illustrative. Template licence: MIT.