∇GradientSchool
Intermediate
15 min read
#generative ai#Guide

Full Generative AI Capstone

Comprehensive guide on Full Generative AI Capstone.

Full Generative AI Capstone

1. Capstone Overview#

This is the final capstone of the Generative AI course.

The objective is to bring together the engineering, modeling, data, evaluation, security, reliability, cost, and product concepts developed throughout the course.

You will design a production-oriented Generative AI platform/application rather than building only a simple chatbot.

The capstone combines:

Architecture & Data Flow
Data
 |
 v
Models
 |
 v
Post-Training
 |
 v
RAG
 |
 v
Reasoning
 |
 v
Agents
 |
 v
Multimodal AI
 |
 v
Enterprise Architecture
 |
 v
Security
 |
 v
FinOps
 |
 v
Reliability
 |
 v
Evaluation
 |
 v
Production

The central objective is:

Design an AI system that is useful, measurable, secure, reliable, cost-aware, and deployable.

2. Capstone Learning Objectives

By completing this capstone, you should be able to:

  1. Translate a real business or educational problem into an AI system specification.
  2. Select appropriate foundation models and supporting models.
  3. Design an enterprise-grade data pipeline.
  4. Build a retrieval-augmented generation system.
  5. Integrate reasoning and verification where appropriate.
  6. Build safe agentic workflows.
  7. Support multimodal inputs where required.
  8. Design model routing and provider abstraction.
  9. Implement evaluation and regression testing.
  10. Design security controls and red-team tests.
  11. Define reliability SLOs and failure-handling strategies.
  12. Model AI costs and establish FinOps controls.
  13. Design observability for AI quality, performance, cost, and safety.
  14. Design deployment and scaling architecture.
  15. Explain architectural trade-offs.
  16. Produce a complete production-readiness plan.

3. Recommended Capstone

The recommended project is:

Enterprise AI Learning & Knowledge Platform

The platform can serve:

  • students
  • teachers
  • administrators
  • enterprise learners
  • support teams
  • knowledge workers

It combines educational and enterprise AI patterns so that the architecture is broad enough to demonstrate the complete course.

4. Product Vision

A user should be able to:

text
Ask questions
Upload documents
Search knowledge
Learn from lessons
Generate practice questions
Analyze images
Ask about audio/video content
Use approved tools
Receive personalized explanations
Track learning progress

Teachers or administrators should be able to:

text
Upload content
Create courses
Review AI-generated material
Monitor usage
Review evaluations
Manage permissions
Inspect AI activity

5. Core Product Requirements

The platform should support:

User capabilities#

  • authentication
  • profile
  • conversations
  • document upload
  • knowledge search
  • personalized learning
  • multimodal questions
  • citations
  • feedback

Teacher capabilities#

  • content management
  • lesson creation
  • question generation
  • document indexing
  • student progress review
  • AI configuration

Administrator capabilities#

  • tenant management
  • policy management
  • usage analytics
  • cost controls
  • audit logs
  • model configuration

6. Functional Architecture

Architecture & Data Flow
 USERS
 |
 v
 Web / Mobile / API
 |
 v
 API Gateway
 |
 v
 AI Application
 |
 +------------------+------------------+
 | | |
 v v v
 Chat RAG Agents
 | | |
 +------------------+------------------+
 |
 v
 AI Gateway
 |
 +----------------+----------------+
 | | |
 v v v
 Model A Model B Private Model
 | | |
 +----------------+----------------+
 |
 v
 Validators
 |
 v
 User

7. Non-Functional Requirements

Define requirements for:

text
Availability
Latency
Security
Privacy
Scalability
Cost
Observability
Maintainability
Auditability
Disaster recovery

Example:

Mathematical Formulation
Availability: >= 99.9%
P95 interactive latency: <= 3 seconds
Critical security incidents: 0
RAG groundedness: >= 95%

These are example targets. Adjust them to your use case.

8. Step 1 — Problem Definition

Before writing code, define:

text
Problem
Target users
Primary workflows
Business value
AI capabilities
Non-AI capabilities
Constraints
Success criteria

Avoid starting with:

"I want to build an AI chatbot."

Start with:

"I want to reduce the time required to answer verified questions about an organization's knowledge base."

9. Problem Statement Template

text
Problem:
Who experiences it?

Current process:
How is it handled today?

Pain:
What is inefficient?

AI opportunity:
What can AI improve?

Constraints:
What must not happen?

Success:
How will improvement be measured?

10. Step 2 — User Personas

Create explicit personas.

Example:

Architecture & Data Flow
Student
 |
 +--> Ask question
 +--> Upload notes
 +--> Practice
 +--> Review progress

Teacher
 |
 +--> Upload content
 +--> Create lesson
 +--> Generate questions
 +--> Review AI output

Administrator
 |
 +--> Manage users
 +--> Manage policies
 +--> Monitor usage
 +--> Review costs

11. Step 3 — Use-Case Prioritization

Classify use cases:

text
High value / Low complexity
High value / High complexity
Low value / Low complexity
Low value / High complexity

Prioritize:

text
High value
+
Manageable complexity
+
Measurable outcome

12. Recommended MVP

Do not build everything at once.

Start with:

Architecture & Data Flow
Authentication
 |
 v
Chat
 |
 v
Document upload
 |
 v
RAG
 |
 v
Citations
 |
 v
Feedback
 |
 v
Evaluation

Then add:

text
Agents
Multimodal
Personalization
Teacher workflows
Advanced analytics

13. Step 4 — Data Architecture

A production AI platform requires more than model prompts.

Architecture & Data Flow
Sources
 |
 +--> PDFs
 +--> Web pages
 +--> Markdown
 +--> Office files
 +--> Images
 +--> Audio
 +--> Video
 |
 v
Ingestion
 |
 v
Validation
 |
 v
Normalization
 |
 v
Chunking / Extraction
 |
 v
Embeddings / Indexing
 |
 v
Retrieval

14. Data Layers

A useful architecture:

Architecture & Data Flow
Raw
 |
 v
Validated
 |
 v
Curated
 |
 v
Derived
 |
 +--> Chunks
 +--> Embeddings
 +--> Metadata
 +--> Summaries
 +--> Evaluation datasets

Keep raw source data recoverable where appropriate.

15. Data Contracts

Define a document contract:

🐍 Python
document = {
 "document_id": "doc-123",
 "tenant_id": "tenant-1",
 "source": "course-material",
 "title": "Lesson 1",
 "version": 3,
 "permissions": ["teacher", "student"],
 "created_at": "...",
 "updated_at": "...",
}

Metadata becomes important for authorization, retrieval, lineage, and auditing.

16. Step 5 — Document Processing

For PDFs and documents:

Architecture & Data Flow
Upload
 |
 v
File validation
 |
 v
Text extraction
 |
 v
OCR if required
 |
 v
Layout / table extraction
 |
 v
Cleaning
 |
 v
Chunking
 |
 v
Embedding
 |
 v
Index

Preserve source references so generated answers can cite evidence.

17. Step 6 — Multimodal Data

For images:

Architecture & Data Flow
Image
 |
 v
Vision model / OCR
 |
 v
Structured representation
 |
 v
Index / context

For audio:

Architecture & Data Flow
Audio
 |
 v
Speech recognition
 |
 v
Speaker / timestamp metadata
 |
 v
Index

For video:

Architecture & Data Flow
Video
 |
 +--> Audio
 +--> Frames
 +--> OCR
 +--> Scene metadata
 |
 v
Unified representation

18. Step 7 — RAG Architecture

A production RAG system:

Architecture & Data Flow
User Query
 |
 v
Authentication
 |
 v
Authorization
 |
 v
Query understanding
 |
 v
Hybrid retrieval
 |
 +--> Dense search
 +--> Sparse search
 |
 v
Reranking
 |
 v
Context construction
 |
 v
LLM
 |
 v
Grounding / citation validation
 |
 v
Response

19. Retrieval Security

Never use the model to decide whether a user may access a document.

Use:

Architecture & Data Flow
User identity
 |
 v
Authorization
 |
 v
Metadata filters
 |
 v
Retrieval
 |
 v
Allowed context

Tenant and permission boundaries must exist outside the model.

20. Step 8 — Model Strategy

Do not assume one model should handle everything.

Possible model ecosystem:

Architecture & Data Flow
Small model
 |
 +--> Classification
 +--> Routing
 +--> Simple extraction

General LLM
 |
 +--> Chat
 +--> Summarization

Reasoning model
 |
 +--> Complex analysis

Vision model
 |
 +--> Images / documents

Speech model
 |
 +--> Audio

Embedding model
 |
 +--> Retrieval

21. Model Gateway

Create an abstraction:

Architecture & Data Flow
Application
 |
 v
AI Gateway
 |
 +--> Model routing
 +--> Authentication
 +--> Quotas
 +--> Cost tracking
 +--> Provider health
 +--> Fallback
 |
 v
Models

The application should not depend tightly on one provider.

22. Model Routing

Example policy:

Architecture & Data Flow
Simple question
 |
 v
Small model

Complex question
 |
 v
General model

High-risk / difficult
 |
 v
Reasoning model

Private data
 |
 v
Approved private model

Routing should consider:

  • capability
  • cost
  • latency
  • privacy
  • availability
  • sovereignty

23. Step 9 — Prompt Management

Prompts should be versioned.

Architecture & Data Flow
Prompt v1
 |
 v
Evaluation
 |
 v
Prompt v2
 |
 v
Evaluation
 |
 v
Production

Store:

  • prompt ID
  • version
  • owner
  • use case
  • model compatibility
  • evaluation results

24. Step 10 — Reasoning and Verification

For tasks requiring higher correctness:

Architecture & Data Flow
User task
 |
 v
Generate
 |
 v
Verify
 |
 +--> Correct --> Return
 |
 +--> Incorrect --> Revise

Use deterministic verifiers where possible.

Examples:

  • Python execution
  • SQL validation
  • schema validation
  • test suites
  • business rules

25. Step 11 — Agent Architecture

Use agents only where dynamic action is genuinely useful.

Architecture & Data Flow
User
 |
 v
Agent
 |
 +--> Search
 +--> Database
 +--> Calendar
 +--> Ticketing
 +--> Content tools
 |
 v
Verifier
 |
 v
Response

Every tool should have explicit permissions.

26. Agent Policy

Define:

text
Allowed tools
Maximum steps
Maximum tool calls
Maximum runtime
Maximum cost
Approval requirements

Example:

🐍 Python
agent_policy = {
 "max_steps": 10,
 "max_tool_calls": 6,
 "max_runtime_seconds": 60,
 "requires_approval_for": [
 "send_email",
 "delete_content",
 ],
}

27. Human-in-the-Loop

High-impact actions should support approval:

Architecture & Data Flow
Agent
 |
 v
Proposed action
 |
 v
Policy
 |
 +--> Low risk --> Execute
 |
 +--> Medium risk --> User approval
 |
 +--> High risk --> Administrator approval

28. Step 12 — Evaluation Architecture

Evaluation should exist before production.

Architecture & Data Flow
Golden Dataset
 |
 v
System
 |
 v
Evaluator
 |
 +--> Quality
 +--> Groundedness
 +--> Safety
 +--> Tool correctness
 +--> Cost
 +--> Latency
 |
 v
Score
 |
 v
Release gate

29. Evaluation Dataset

Include:

text
Normal examples
Edge cases
Long inputs
Ambiguous questions
No-answer cases
Security attacks
Multilingual cases
Multimodal cases
Tool failures

Organize by slices.

30. RAG Evaluation

Track:

text
Recall@K
Precision@K
MRR
NDCG
Groundedness
Citation correctness
Citation completeness
No-evidence behavior

Do not evaluate only final answer quality.

31. Agent Evaluation

Measure:

text
Task success
Tool selection
Tool arguments
Trajectory length
Recovery
Safety
Cost
Latency

A successful final answer with unsafe intermediate actions is not a successful agent.

32. Multimodal Evaluation

Possible metrics:

text
OCR accuracy
Speech WER
Image classification
Visual question answering
Temporal video accuracy
Spatial grounding
Cross-modal retrieval

Use task-specific metrics.

33. Step 13 — Security Architecture

Architecture & Data Flow
User
 |
 v
Identity
 |
 v
Authorization
 |
 v
AI Gateway
 |
 v
Policy Engine
 |
 +--> Model
 +--> RAG
 +--> Tools
 |
 v
Output validation
 |
 v
DLP / audit

Security should be enforced outside the model.

34. Red-Team Program

Test:

text
Prompt injection
Indirect injection
RAG poisoning
Cross-tenant access
Tool abuse
Data exfiltration
Jailbreaks
Multimodal injection
Economic DoS
Agent loops

Every important vulnerability becomes a regression test.

35. Step 14 — Reliability Architecture

Use:

text
Timeouts
Retries
Backoff
Circuit breakers
Fallbacks
Rate limits
Queues
Backpressure
Load shedding
Health checks

For agents add:

text
Step limits
Tool limits
Budget limits
Checkpoints
Human escalation

36. Reliability SLOs

Define measurable objectives.

Example:

Mathematical Formulation
Availability >= 99.9%
P95 latency <= 3 sec
Tool success >= 99%
RAG groundedness >= 95%
Critical safety failures = 0

Use error budgets to manage reliability work.

37. Step 15 — AI FinOps

Track:

text
Input tokens
Output tokens
Model calls
Embedding calls
Reranking
Storage
GPU time
Tool usage
Evaluation

A useful metric:

›Cost per successful task

not only:

›Cost per request

38. Cost Controls

Implement:

text
Tenant budgets
Rate limits
Token limits
Model routing
Caching
Batching
Prompt compression
Context limits

Example:

🐍 Python
if estimated_cost > tenant_budget:
 raise RuntimeError("AI budget exceeded")

39. Step 16 — Observability

Use:

text
Logs
Metrics
Traces
Quality signals
Cost signals
Security signals

A request should have a correlation ID.

🐍 Python
event = {
 "request_id": "req-123",
 "tenant_id": "tenant-1",
 "model": "model-x",
 "latency_ms": 1200,
 "input_tokens": 1800,
 "output_tokens": 300,
}

Avoid logging sensitive content unnecessarily.

40. Step 17 — Database Architecture

Typical components:

Architecture & Data Flow
PostgreSQL
 |
 +--> Users
 +--> Tenants
 +--> Courses
 +--> Lessons
 +--> Permissions
 +--> Conversations
 +--> Feedback
 +--> Audit records

Vector store
 |
 +--> Chunks
 +--> Embeddings
 +--> Metadata

Object storage
 |
 +--> PDFs
 +--> Images
 +--> Audio
 +--> Video

Keep authoritative business data separate from derived AI indexes.

41. Multi-Tenancy

A tenant-aware architecture:

Architecture & Data Flow
Request
 |
 v
Tenant identity
 |
 v
Authorization
 |
 +--> SQL tenant filter
 +--> Vector tenant filter
 +--> Object storage scope
 +--> Cache scope
 +--> Memory scope

Every data access path should be tenant-aware.

42. Step 18 — API Architecture

Example services:

text
/api/auth
/api/users
/api/courses
/api/documents
/api/search
/api/chat
/api/agents
/api/evaluations
/api/feedback
/api/admin

Keep business APIs separate from AI orchestration where useful.

43. Example FastAPI Pattern

🐍 Python
from fastapi import FastAPI

app = FastAPI()

@app.post("/chat")
def chat(request: dict):
 user = authenticate(request)
 authorize(user, "chat")

 result = ai_service.answer(
 tenant_id=user["tenant_id"],
 message=request["message"],
 )

 return result

Production systems should use typed request/response models, centralized authentication, validation, observability, and error handling.

44. Step 19 — Async Processing

Use asynchronous jobs for:

text
Video processing
Large document ingestion
Embedding
Batch evaluation
Synthetic data generation
Large-scale indexing

Architecture:

Architecture & Data Flow
API
 |
 v
Queue
 |
 +--> Worker
 +--> Worker
 +--> Worker
 |
 v
Result store

45. Step 20 — Deployment Architecture

A production architecture might use:

Architecture & Data Flow
Internet
 |
 v
WAF / Load Balancer
 |
 v
API Gateway
 |
 v
Application services
 |
 +--> PostgreSQL
 +--> Object storage
 +--> Vector DB
 +--> Queue
 |
 v
AI Gateway
 |
 +--> External models
 +--> Private models
 +--> GPU inference

Containerization and orchestration can be added according to scale.

46. GPU Serving

For self-hosted models:

Architecture & Data Flow
AI Gateway
 |
 v
Inference router
 |
 +--> GPU worker 1
 +--> GPU worker 2
 +--> GPU worker 3

Consider:

  • batching
  • KV cache
  • quantization
  • GPU utilization
  • autoscaling
  • model loading time

47. Model Serving Strategy

Choose among:

text
External API
Private cloud
On-premises
Edge
Hybrid

Decision factors:

text
Privacy
Cost
Latency
Capacity
Sovereignty
Compliance
Operational maturity

48. Step 21 — CI/CD for AI

A strong pipeline:

Architecture & Data Flow
Code change
 |
 v
Unit tests
 |
 v
Security tests
 |
 v
AI evaluation
 |
 v
Cost check
 |
 v
Performance check
 |
 v
Canary
 |
 v
Production

An AI deployment should not depend only on software unit tests.

49. Model and Prompt Versioning

Version:

text
Application
Model
Prompt
Embedding model
Reranker
RAG configuration
Tool schema
Policy
Evaluation dataset

A production result should be reproducible from these versions.

50. Step 22 — Canary Deployment

Example:

›Old version: 95%
New version: 5%

Compare:

text
Error rate
Latency
Quality
Safety
Cost

Expand only when results meet release criteria.

51. Step 23 — Disaster Recovery

Define:

›RTO
RPO

Back up or reproduce:

text
Business data
Documents
Model artifacts
Configuration
Prompts
Indexes
Evaluation datasets

Do not assume a vector database is the authoritative source of knowledge.

52. Step 24 — Product Analytics

Measure:

text
Active users
Requests
Successful tasks
User feedback
Retention
Feature usage
Escalations
Cost

AI quality metrics should connect to product outcomes.

53. Business Metrics

For an enterprise assistant:

text
Time saved
Resolution rate
Human escalation rate
Cost per resolved case
User satisfaction
Knowledge retrieval success

The strongest capstone connects:

text
AI metrics
+
business metrics

54. Step 25 — Architecture Review

Review the system across:

AreaQuestions
ProductDoes it solve a real problem?
DataIs data reliable and authorized?
ModelsAre models appropriate?
RAGIs retrieval accurate and secure?
AgentsAre actions bounded?
MultimodalAre inputs processed correctly?
SecurityCan attackers abuse it?
ReliabilityWhat happens when dependencies fail?
CostIs the system economically viable?
EvaluationCan quality be measured?
OperationsCan the team operate it?

55. End-to-End Architecture

Architecture & Data Flow
 USERS
 |
 v
 WEB / MOBILE / API
 |
 v
 WAF / GATEWAY
 |
 v
 IDENTITY + AUTHORIZATION
 |
 v
 AI APPLICATION
 |
 +-----------------------+-----------------------+
 | | |
 v v v
 CHAT RAG AGENTS
 | | |
 | +-------+-------+ |
 | | | |
 | v v |
 | Retrieval Reranking |
 | | | |
 | +-------+-------+ |
 | | |
 +-----------------------+-----------------------+
 |
 v
 AI GATEWAY
 |
 +------------------------+------------------------+
 | | |
 v v v
 Small Models General Models Reasoning Models
 | | |
 +------------------------+------------------------+
 |
 v
 VALIDATION / POLICY
 |
 +-----------+-----------+
 | |
 v v
 TOOLS DLP
 | |
 +-----------+-----------+
 |
 v
 RESPONSE

 DATA PLANE
 ---------------------------------------------------
 PostgreSQL | Vector DB | Object Storage | Queue

 CONTROL PLANE
 ---------------------------------------------------
 Models | Prompts | Policies | Evaluations | Budgets

 OPERATIONS
 ---------------------------------------------------
 Logs | Metrics | Traces | Quality | Security | Cost

56. Suggested Technology Stack

One possible implementation:

Frontend#

›React / Next.js

API#

›FastAPI
Python

Database#

›PostgreSQL

Vector search#

›pgvector / dedicated vector database

Object storage#

›S3-compatible storage

Queue#

›Redis / RabbitMQ / Kafka

AI orchestration#

›LangChain / LangGraph

Model serving#

›vLLM / equivalent inference server

Observability#

text
OpenTelemetry
Prometheus
Grafana

The exact technology choices are less important than demonstrating the architectural principles.

57. Recommended Repository Structure

Architecture & Data Flow
genai-capstone/
|
+-- apps/
| +-- api/
| +-- web/
|
+-- services/
| +-- ai_gateway/
| +-- rag/
| +-- agents/
| +-- evaluation/
| +-- ingestion/
|
+-- models/
| +-- routing/
| +-- prompts/
|
+-- data/
| +-- schemas/
| +-- evaluation/
|
+-- security/
| +-- policies/
| +-- red_team/
|
+-- infra/
| +-- docker/
| +-- kubernetes/
|
+-- tests/
| +-- unit/
| +-- integration/
| +-- evaluation/
| +-- security/
|
+-- docs/
| +-- architecture/
| +-- runbooks/
|
+-- README.md

58. Recommended Service Boundaries

Do not split everything into microservices immediately.

A practical starting point:

Architecture & Data Flow
API Application
 |
 +--> AI Gateway module
 +--> RAG module
 +--> Agent module
 +--> Evaluation module
 +--> Ingestion worker

Split services when:

  • scaling differs
  • ownership differs
  • failure isolation is needed
  • deployment independence matters

59. Security Threat Model

At minimum, test:

Architecture & Data Flow
User
 |
 +--> Prompt injection
 +--> Data exfiltration
 +--> Unauthorized retrieval
 +--> Tool abuse
 +--> Rate abuse
 |
 v
AI Platform
 |
 +--> RAG poisoning
 +--> Cache leakage
 +--> Tenant leakage
 +--> Supply-chain risk

60. Reliability Threat Model

Test:

text
Model outage
Vector DB outage
Database outage
Queue saturation
Tool outage
Network latency
GPU failure
Provider rate limit
Agent loop

For every failure ask:

text
Detect?
Contain?
Fallback?
Recover?
Measure?

61. Cost Threat Model

Test:

text
Long prompts
Long outputs
Repeated requests
Agent loops
Expensive model routing
Large multimodal inputs
Evaluation spikes
Embedding spikes

Build controls before these become production incidents.

62. Evaluation Threat Model

Test for:

text
Model regression
Prompt regression
RAG regression
Tool regression
Security regression
Latency regression
Cost regression

Every major change should trigger the appropriate evaluation suite.

63. Capstone Milestones

A recommended sequence:

text
Milestone 1
Problem + architecture

Milestone 2
Data + ingestion

Milestone 3
RAG

Milestone 4
Model gateway

Milestone 5
Evaluation

Milestone 6
Agent workflows

Milestone 7
Security

Milestone 8
Reliability

Milestone 9
FinOps

Milestone 10
Deployment

Milestone 11
Observability

Milestone 12
Final demo

64. Milestone 1 Deliverables

Produce:

text
Problem statement
User personas
Use cases
Requirements
Architecture diagram
Threat model
Success metrics

65. Milestone 2 Deliverables

Produce:

text
Data schema
Ingestion pipeline
Validation rules
Metadata design
Chunking strategy
Indexing strategy
Data lineage

66. Milestone 3 Deliverables

Produce:

text
RAG pipeline
Retrieval strategy
Reranking
Authorization
Citation generation
Groundedness evaluation

67. Milestone 4 Deliverables

Produce:

text
AI gateway
Model registry
Routing policy
Provider abstraction
Fallback
Cost tracking
Health checks

68. Milestone 5 Deliverables

Produce:

text
Golden dataset
Evaluation scripts
Quality metrics
RAG metrics
Safety tests
Regression suite
Release gate

69. Milestone 6 Deliverables

Produce:

text
Agent graph
Tool registry
Tool schemas
Policy engine
Execution limits
Human approval
Agent evaluation

70. Milestone 7 Deliverables

Produce:

text
Threat model
Security controls
Prompt-injection tests
Tenant-isolation tests
Tool-abuse tests
DLP controls
Audit logging

71. Milestone 8 Deliverables

Produce:

text
SLOs
Timeouts
Retries
Circuit breakers
Fallbacks
Load tests
Chaos tests
Incident runbooks

72. Milestone 9 Deliverables

Produce:

text
Cost model
Tenant budgets
Usage dashboard
Model routing economics
Caching strategy
Cost anomaly detection

73. Milestone 10 Deliverables

Produce:

text
Container configuration
Deployment manifests
CI/CD pipeline
Canary release
Rollback
Disaster recovery plan

74. Milestone 11 Deliverables

Produce:

text
Logs
Metrics
Traces
AI quality dashboard
Cost dashboard
Security dashboard
Alert rules

75. Milestone 12 — Final Demo

The final demonstration should show a complete workflow:

Architecture & Data Flow
User
 |
 v
Authenticate
 |
 v
Ask question
 |
 v
Retrieve authorized evidence
 |
 v
Model reasoning
 |
 v
Generate answer
 |
 v
Validate citations
 |
 v
Return response
 |
 v
Record evaluation / telemetry

Then demonstrate at least one:

text
Agent workflow
Multimodal workflow
Failure scenario
Security attack
Fallback

76. Required Capstone Features

A strong submission should include:

text
[ ] Authentication
[ ] Authorization
[ ] Multi-tenancy
[ ] Document ingestion
[ ] RAG
[ ] Citations
[ ] Model gateway
[ ] Model routing
[ ] Evaluation
[ ] Security tests
[ ] Agent workflow
[ ] Tool validation
[ ] Reliability controls
[ ] Cost tracking
[ ] Observability
[ ] CI/CD
[ ] Documentation

77. Advanced Optional Features

For an exceptional submission, add:

text
[ ] Multimodal RAG
[ ] Voice interface
[ ] Video understanding
[ ] Reasoning model routing
[ ] Semantic cache
[ ] Prefix caching
[ ] Self-hosted model
[ ] GPU inference
[ ] Knowledge graph
[ ] Adaptive learning
[ ] Persistent memory
[ ] Human approval console
[ ] Automated red-team pipeline
[ ] AI FinOps dashboard
[ ] Multi-region deployment

78. Capstone Evaluation Rubric

CategoryWeight
Problem definition5%
Architecture10%
Data engineering10%
RAG10%
Model strategy10%
Agents / workflows10%
Evaluation10%
Security10%
Reliability10%
FinOps5%
Observability5%
Documentation / demo5%

Total:

›100%

79. Architecture Quality Rubric

Excellent#

text
Clear boundaries
Explicit trade-offs
Security by design
Evaluation-driven
Observable
Recoverable
Cost-aware

Weak#

text
One large application
No evaluation
No authorization
No failure strategy
No cost model
No monitoring

80. RAG Quality Rubric

Evaluate:

text
Retrieval accuracy
Permission correctness
Freshness
Groundedness
Citation correctness
No-answer behavior
Latency

81. Agent Quality Rubric

Evaluate:

text
Task completion
Tool selection
Argument correctness
Safety
Step efficiency
Recovery
Human escalation
Cost

82. Security Rubric

A strong system should demonstrate:

text
Authentication
Authorization
Tenant isolation
Least privilege
Input validation
Output validation
Prompt-injection defense
Tool security
DLP
Auditability
Red-team testing

83. Reliability Rubric

Demonstrate:

text
Timeouts
Retry policies
Fallbacks
Circuit breakers
Load handling
SLOs
Monitoring
Incident response
Recovery

84. FinOps Rubric

Demonstrate:

text
Cost attribution
Budget controls
Model routing
Token monitoring
Caching
Usage analytics
Cost optimization

85. Final Architecture Review Questions

Before declaring the project complete, answer:

  1. Why does this system need AI?
  2. Why did you select these models?
  3. Why did you choose RAG?
  4. Where is authorization enforced?
  5. What happens when retrieval fails?
  6. What happens when the model fails?
  7. What happens when a tool fails?
  8. How do you prevent agent loops?
  9. How do you prevent tenant leakage?
  10. How do you measure answer quality?
  11. How do you measure groundedness?
  12. How do you detect security attacks?
  13. How much does one successful task cost?
  14. How does the system scale?
  15. What are the SLOs?
  16. What happens during an outage?
  17. How do you roll back a model?
  18. How do you reproduce a production response?
  19. How do you protect user data?
  20. What would you improve in version 2?

86. Final Capstone Project Variations

The same architecture can be adapted to other domains.

Enterprise Knowledge Assistant#

text
Documents
+
Policies
+
Enterprise search
+
Workflow agents

Customer Support AI#

text
Knowledge base
+
CRM
+
Ticketing
+
Agent escalation

Financial Research Assistant#

text
Documents
+
Market data
+
SQL
+
Research tools
+
Verification

Healthcare Knowledge Assistant#

text
Clinical documents
+
Search
+
Structured information
+
Strong safety controls

Use additional domain-specific governance for high-risk domains.

87. Educational AI Variant

For an educational platform:

Architecture & Data Flow
Student
 |
 v
Tutor
 |
 +--> Curriculum
 +--> RAG
 +--> Student memory
 +--> Reasoning
 +--> Assessment
 +--> Multimodal content
 |
 v
Personalized learning
 |
 v
Assessment
 |
 v
Student model update

The system can support:

  • lesson explanations
  • practice questions
  • adaptive difficulty
  • document understanding
  • image-based questions
  • voice tutoring
  • video summaries
  • teacher dashboards

88. Educational AI Safety

Include:

text
Age-appropriate behavior
Student privacy
Teacher oversight
Academic-integrity controls
Content moderation
Human escalation
Audit logs

The AI should support learning rather than simply optimize for answer completion.

89. Production Readiness Checklist

Product#

text
[ ] Clear user problem
[ ] Measurable outcomes
[ ] User feedback

Data#

text
[ ] Data contracts
[ ] Validation
[ ] Lineage
[ ] Permissions
[ ] Versioning

AI#

text
[ ] Model selection
[ ] Prompt versioning
[ ] RAG
[ ] Routing
[ ] Verification

Agents#

text
[ ] Tool allowlist
[ ] Authorization
[ ] Step limits
[ ] Human approval

Security#

text
[ ] Threat model
[ ] Red team
[ ] DLP
[ ] Tenant isolation
[ ] Audit

Reliability#

text
[ ] SLOs
[ ] Timeouts
[ ] Retries
[ ] Fallback
[ ] Disaster recovery

FinOps#

text
[ ] Cost model
[ ] Budgets
[ ] Quotas
[ ] Cost monitoring

Evaluation#

text
[ ] Golden dataset
[ ] Regression tests
[ ] RAG evaluation
[ ] Safety evaluation
[ ] Agent evaluation

Operations#

text
[ ] Logs
[ ] Metrics
[ ] Traces
[ ] Alerts
[ ] Runbooks

90. Final Mental Model

The complete Generative AI engineering journey can be represented as:

Architecture & Data Flow
 USER / BUSINESS
 |
 v
 PRODUCT PROBLEM
 |
 v
 DATA FOUNDATION
 |
 v
 MODEL FOUNDATION
 |
 +------------------+------------------+
 | | |
 v v v
 RAG REASONING TOOLS
 | | |
 +------------------+------------------+
 |
 v
 AGENTS
 |
 v
 MULTIMODAL AI
 |
 v
 AI GATEWAY
 |
 +-----------------------+-----------------------+
 | | |
 v v v
 SECURITY RELIABILITY FINOPS
 | | |
 +-----------------------+-----------------------+
 |
 v
 EVALUATION
 |
 v
 OBSERVABILITY
 |
 v
 CI / CD
 |
 v
 PRODUCTION
 |
 v
 USER FEEDBACK
 |
 +----------+
 |
 v
 IMPROVEMENT

The complete system is a continuous engineering loop:

Architecture & Data Flow
Build
 |
 v
Evaluate
 |
 v
Secure
 |
 v
Deploy
 |
 v
Observe
 |
 v
Learn
 |
 v
Improve
 |
 +---------> Build

The deepest lesson of the course is:

Generative AI engineering is not primarily about calling a large language model. It is about designing a complete system around intelligence: data, models, retrieval, reasoning, tools, evaluation, security, reliability, cost, and human outcomes.

91. Key Takeaways

  1. A production Generative AI application is a system, not just a model.
  2. Start with a real problem and measurable outcome.
  3. Data quality and authorization strongly influence AI quality.
  4. RAG is an architectural pattern for connecting models to controlled external knowledge.
  5. Models should be selected according to task, privacy, latency, cost, and reliability requirements.
  6. A model gateway creates flexibility and centralized controls.
  7. Reasoning should be paired with verification when correctness matters.
  8. Agents should have explicit tools, policies, limits, and approval mechanisms.
  9. Multimodal systems require modality-specific processing and evaluation.
  10. Security must be enforced outside the model.
  11. Multi-tenancy must cover databases, vector search, object storage, caches, memory, and tools.
  12. Evaluation should be part of the development and deployment lifecycle.
  13. AI systems need specialized reliability mechanisms.
  14. Cost should be measured per useful outcome, not only per API request.
  15. Observability should include technical, quality, cost, and security signals.
  16. Model, prompt, data, tool, policy, and evaluation versions should be traceable.
  17. AI CI/CD should include evaluation and security gates.
  18. Production systems should support rollback and disaster recovery.
  19. Human oversight remains important for high-impact workflows.
  20. Good AI architecture balances capability with safety, reliability, economics, and maintainability.
  21. The strongest capstone is not the one with the most AI features; it is the one that demonstrates sound engineering decisions.
  22. The ultimate objective is measurable value for users and the organization.

92. Knowledge Check

Question 1#

What is the main purpose of the capstone?

Answer: To integrate the course's AI, data, RAG, agents, multimodal, security, reliability, evaluation, cost, and production engineering concepts into one coherent system.

Question 2#

Why should the capstone begin with a problem statement?

Answer: Because technology should serve a measurable user or business outcome rather than becoming an end in itself.

Question 3#

Why is authorization required before retrieval?

Answer: Because the model should never be trusted to determine whether a user is allowed to access enterprise information.

Question 4#

Why should a model gateway exist?

Answer: It provides a common control point for routing, provider abstraction, policies, cost tracking, health monitoring, and fallback.

Question 5#

Why are agents given explicit execution limits?

Answer: To prevent loops, excessive tool usage, runaway costs, and unsafe long-running behavior.

Question 6#

What should be evaluated before deploying a model or prompt change?

Answer: Relevant quality, RAG, safety, latency, cost, and agent/tool regression tests.

Question 7#

Why is observability important?

Answer: Production AI systems are probabilistic and distributed, so teams need visibility into technical behavior, quality, cost, and security.

Question 8#

What is a good definition of production-ready AI?

Answer: AI that provides useful outcomes within defined quality, security, reliability, latency, cost, governance, and operational requirements.

Question 9#

What is the most important security principle?

Answer: Never rely on the model alone to enforce authorization, permissions, or other critical security boundaries.

Question 10#

What is the final mental model for Generative AI engineering?

Answer: Build an integrated system around models, continuously evaluate it, secure it, observe it, operate it reliably, control its economics, and improve it using real evidence.

93. Full Course Mental Map

The complete learning journey can now be viewed as:

Architecture & Data Flow
MACHINE LEARNING
 |
 v
SUPERVISED LEARNING
 |
 v
TIME SERIES
 |
 v
GENERATIVE AI
 |
 +--> LLM Foundations
 |
 +--> Transformers
 |
 +--> RAG
 |
 +--> Agents
 |
 +--> Evaluation
 |
 +--> Multimodal AI
 |
 +--> Fine-Tuning
 |
 +--> Open / Sovereign Models
 |
 +--> LLMOps
 |
 +--> End-to-End Applications
 |
 +--> Security
 |
 +--> Advanced RAG
 |
 +--> AI Platform Engineering
 |
 +--> Distributed Inference
 |
 +--> Data Engineering
 |
 +--> Advanced Evaluation
 |
 +--> Synthetic Data
 |
 +--> Distillation
 |
 +--> Advanced Training
 |
 +--> Post-Training
 |
 +--> Reasoning
 |
 +--> Small Language Models
 |
 +--> Computer Use
 |
 +--> Advanced Multimodal
 |
 +--> Generative AI for Code
 |
 +--> Enterprise AI
 |
 +--> AI FinOps
 |
 +--> AI Reliability
 |
 +--> AI Red Teaming
 |
 +--> Future Architectures
 |
 v
FULL GENERATIVE AI CAPSTONE

94. Final Capstone Submission

A complete submission should contain:

text
1. README
2. Architecture document
3. Source code
4. Data pipeline
5. RAG implementation
6. Model gateway
7. Agent workflow
8. Evaluation suite
9. Security tests
10. Reliability tests
11. Cost model
12. Observability dashboards
13. Deployment configuration
14. Runbooks
15. Demo
16. Final technical report

The final report should explain not only:

›"What did you build?"

but also:

›"Why did you build it this way?"

and:

›"What evidence shows that it works?"

95. Final Reflection

After completing the capstone, write a short reflection covering:

text
What was the hardest engineering problem?

Which architectural decision had the biggest impact?

Which assumption turned out to be wrong?

What security weakness did you discover?

What reliability failure did you discover?

What was the largest cost driver?

Which evaluation metric mattered most?

What would you change in version 2?

What did this project teach you about production AI?

This reflection is part of the engineering learning process.

96. Course Completion

You have now reached the end of the Generative AI roadmap.

The progression is:

Architecture & Data Flow
Understand models
 |
 v
Build applications
 |
 v
Connect knowledge
 |
 v
Add reasoning
 |
 v
Add tools and agents
 |
 v
Add multimodal capability
 |
 v
Evaluate
 |
 v
Secure
 |
 v
Optimize cost
 |
 v
Engineer reliability
 |
 v
Deploy
 |
 v
Operate
 |
 v
Improve

The next step is no longer another notebook.

The next step is to build.

Learn the architecture. Implement the system. Measure it. Break it safely. Fix it. Deploy it. Then improve it using evidence.

Knowledge Checkpoint

Full GenAI Capstone Architecture Checkpoint

Q1.What architectural components constitute a full-stack production Enterprise GenAI application?
AFrontend UI with SSE streaming, API Gateway with Auth/Rate Limiting, Multi-Agent Orchestration (LangGraph), Hybrid Retrieval Engine with Reranker, Guardrails layer, and Observability/Telemetry stack.
BA single Python script with `input()` and `print()` statements.
CAn HTML file with an iframe pointing to ChatGPT.
DA local SQLite database with no web server.
Q2.How is continuous quality regression testing automated before merging PRs in a production GenAI repository?
ACI/CD pipelines automatically execute synthetic evaluation runs against a versioned golden evaluation dataset and assert that Ragas/G-Eval scores do not drop below predefined thresholds.
BBy manually chatting with the model for 5 minutes.
CBy counting the number of lines in pull requests.
DBy running unit tests on CSS styles.
Q3.What is the primary trade-off when selecting between a Managed API (e.g. OpenAI / Anthropic) versus a Self-Hosted Open-Weights model (e.g. vLLM with LLaMA 3.1)?
AManaged APIs provide zero infrastructure management and state-of-the-art reasoning at per-token costs; self-hosted open models provide complete data sovereignty, zero egress privacy, customized fine-tuning, and predictable fixed compute costs at high scale.
BManaged APIs are free; self-hosted models cost millions per day.
CSelf-hosted models cannot process English text.
DThere is no difference in capability or cost.
Track Your Learning

Finished studying this notebook?

Mark this guide as completed to update your course progress roadmap.

Previous Lesson
End-to-End Generative AI Application Projects