Official · AI & data · advanced
Keep enterprise AI search fresh and private
Handle document changes, ACL revocation and deletion across a retrieval pipeline.
The question
Design an internal question-answering product over enterprise documents. Focus on the lifecycle from ingestion to a cited answer, especially when permission or source content changes. Explain a useful degraded response when freshness or authorization cannot be established.
- Ten million documents change at up to 200 updates/second; the product serves 400 questions/second.
- Permissions exist at both workspace and document level. A user can lose access while a retrieval or generation request is running.
- Documents may contain instructions aimed at the assistant. Answers must distinguish source evidence from model assumptions.
What to cover
Show ingestion, versioned chunks/indexes, retrieval, authorization checks and cited answer assembly.
Trace an ACL revoke and a deleted document during an in-flight question; explain cache invalidation and reconciliation.
Define test slices, relevance/grounding/leakage measures, a baseline and launch thresholds.
Size ingest/query load, estimate per-answer cost assumptions and choose a degraded behavior when dependencies fail.
How your practice is reviewed
- Version and authorization lifecycle (35 points): Explains enforcement, race handling, stale indexes/caches and deletion propagation with bounded claims.
- Grounding and evaluation (25 points): Measures retrieval, citation correctness and permission leakage on meaningful slices; treats document instructions as data.
- Capacity and degraded behavior (25 points): Calculations and operating policies support the stated freshness/cost/latency goals.
- Alternatives and assumptions (15 points): Makes retrieval/model choices conditional on evidence and explains a viable alternative.
Discuss & learn
Ask the community, or tag @Coach for a contextual AI answer.