∇GradientSchool
Advanced
210–270 min read
#Generative AI Projects#RAG#Agents#Multimodal AI#FastAPI#LLMOps#Fine-Tuning#Vector Databases#Tool Calling#AI Tutor#Enterprise AI#Production Architecture

End-to-End Generative AI Application Projects

A portfolio-focused notebook that combines the Generative AI concepts learned so far into complete applications, including an educational AI tutor, document intelligence system, multimodal assistant, research agent, and enterprise knowledge assistant.

End-to-End Generative AI Application Projects

1. Introduction#

You have now learned the major building blocks of modern Generative AI:

text
LLMs
Transformers
Prompting
Embeddings
RAG
Vector databases
Agents
LangChain
LangGraph
Multimodal AI
Fine-tuning
LoRA
QLoRA
Open-weight models
Sovereign AI
LLMOps
Production deployment

The next step is to combine them.

Real-world AI engineering rarely looks like:

›Prompt -> Model -> Answer

Instead, it looks more like:

Architecture & Data Flow
User
 |
 v
Application
 |
 v
Authentication
 |
 v
AI Gateway
 |
 +--> Retrieval
 |
 +--> Tools
 |
 +--> Memory
 |
 +--> Model
 |
 v
Validation
 |
 v
Evaluation
 |
 v
Response

This notebook turns the concepts into complete applications.

2. Learning Objectives

By the end of this notebook, you should be able to:

  1. Design an end-to-end GenAI application
  2. Select an appropriate model
  3. Build a RAG pipeline
  4. Add vector search
  5. Add tool calling
  6. Build an agent workflow
  7. Integrate multimodal inputs
  8. Add authentication
  9. Build an API with FastAPI
  10. Add structured outputs
  11. Add caching
  12. Add observability
  13. Evaluate AI quality
  14. Apply model routing
  15. Use fine-tuned adapters
  16. Design multi-tenant systems
  17. Build educational AI applications
  18. Deploy local or cloud models
  19. Implement production safeguards
  20. Build portfolio-grade projects

3. The End-to-End GenAI Stack

A modern application can be represented as:

Architecture & Data Flow
 USER INTERFACE
 |
 v
 API / BACKEND
 |
 v
 AI GATEWAY
 |
 +---------------+---------------+
 | | |
 v v v
 Cache Router Safety
 |
 +-----------+-----------+
 | |
 v v
 RAG Tools
 | |
 +-----------+-----------+
 |
 v
 LLM
 |
 v
 Output Validation
 |
 v
 Application
 |
 v
 Observability
 |
 v
 Evaluation

4. Project Development Lifecycle

Every project should follow:

Architecture & Data Flow
Problem
 |
 v
Requirements
 |
 v
Architecture
 |
 v
Baseline
 |
 v
Prototype
 |
 v
Evaluation
 |
 v
Security
 |
 v
Optimization
 |
 v
Deployment
 |
 v
Monitoring

Do not begin by choosing a model.

Begin by defining the problem.

5. Project 1: Educational AI Tutor

This project is especially useful for an educational platform.

The goal is to build an AI tutor that can:

text
Answer student questions
Explain concepts
Provide hints
Generate examples
Reference course material
Adapt explanations to learner level

6. AI Tutor Requirements

The tutor should support:

text
Student authentication
Course selection
Lesson context
Question answering
RAG
Conversation history
Streaming responses
Usage limits
Safety checks
Teacher-controlled content
Evaluation

7. AI Tutor Architecture

Architecture & Data Flow
 STUDENT
 |
 v
 Web / Mobile UI
 |
 v
 FastAPI
 |
 +-------+-------+
 | |
 v v
 Authentication Student Context
 | |
 +-------+-------+
 |
 v
 AI Gateway
 |
 +----------+----------+
 | |
 v v
 Cache Router
 |
 +------------+------------+
 | | |
 v v v
 Small Medium Large
 Model Model Model
 | | |
 +------------+-------------+
 |
 v
 RAG
 |
 +------------+------------+
 | | |
 v v v
 Lessons PDFs Teacher Content
 |
 v
 Output Validation
 |
 v
 Student

8. AI Tutor Data Model

A simplified schema:

text
Student
------
id
name
school_id
grade

Course
------
id
school_id
title
subject

Lesson
------
id
course_id
title
content

Conversation
-----------
id
student_id
course_id
lesson_id

Message
-------
id
conversation_id
role
content
timestamp

The exact schema depends on the application.

9. Course Content Pipeline

Teacher content can enter the system through:

text
Markdown
PDF
DOCX
Images
Videos

Pipeline:

Architecture & Data Flow
Teacher uploads content
 |
 v
Content extraction
 |
 v
Cleaning
 |
 v
Chunking
 |
 v
Embedding
 |
 v
Vector database

10. Retrieval Pipeline

Student question:

›"Explain photosynthesis."

Pipeline:

Architecture & Data Flow
Question
 |
 v
Embedding
 |
 v
Vector search
 |
 v
Relevant lesson chunks
 |
 v
Optional reranking
 |
 v
Prompt construction
 |
 v
LLM

11. Grounded Tutor Prompt

Conceptually:

text
You are an educational tutor.

Use the supplied course material as the primary source.

Student level:
{student_level}

Lesson:
{lesson}

Question:
{question}

Relevant course material:
{context}

Explain clearly and encourage understanding.
If the material does not contain the answer, say so rather than inventing facts.

The exact prompt should be evaluated rather than assumed to be optimal.

12. Hint Mode

An educational tutor should not always give the complete answer.

Modes could include:

text
Explain
Hint
Example
Practice
Solution

Example:

text
Student:
"I cannot solve this equation."

Mode:
Hint

Tutor:
"What operation could you perform first to isolate the variable?"

This supports learning rather than simply answer delivery.

13. Difficulty Adaptation

The tutor can use:

text
Grade
Course
Lesson
Previous interactions
Teacher settings

to choose an appropriate explanation level.

Conceptually:

Architecture & Data Flow
Same concept
 |
 +---+---+
 | |
 v v
Beginner Advanced
 | |
 v v
Simple Technical

14. Teacher Controls

Teachers should be able to configure:

text
Allowed content
Explanation style
Difficulty
Hint policy
AI availability
Course scope

This creates a boundary between:

›General model knowledge

and:

›Teacher-approved course content

15. AI Tutor API

Example endpoints:

text
POST /api/v1/tutor/chat
POST /api/v1/tutor/hint
POST /api/v1/tutor/explain
GET /api/v1/courses/{course_id}
GET /api/v1/lessons/{lesson_id}

The API should authenticate and authorize every request.

16. Tutor Request Example

json
{
 "course_id": "math-101",
 "lesson_id": "quadratic-equations",
 "message": "How do I solve x² + 5x + 6 = 0?",
 "mode": "hint"
}

The backend can:

Architecture & Data Flow
Validate request
 |
 v
Check student access
 |
 v
Retrieve lesson content
 |
 v
Construct prompt
 |
 v
Generate response
 |
 v
Validate response
 |
 v
Stream to student

17. Project 2: Enterprise Document Intelligence

Build a system that allows users to ask questions about company documents.

Supported sources:

text
PDF
DOCX
Markdown
Images
Spreadsheets

18. Document Intelligence Architecture

Architecture & Data Flow
Documents
 |
 v
Ingestion
 |
 v
Parsing
 |
 v
Chunking
 |
 v
Embeddings
 |
 v
Vector DB
 |
 v
Retriever
 |
 v
LLM
 |
 v
Answer + Citations

19. Metadata

Store metadata such as:

text
document_id
tenant_id
department
author
created_at
version
access_level
page_number

Metadata filtering is essential for enterprise security.

20. Tenant-Aware Retrieval

Suppose:

›Company A
Company B

The query from Company A must never retrieve:

›Company B documents

Conceptually:

🐍 Python
results = vector_db.search(
 query_embedding,
 filters={
 "tenant_id": current_tenant
 }
)

Authorization should be enforced independently of retrieval where appropriate.

21. Citation Generation

A useful enterprise response:

text
Answer:
The company provides 30 days of annual leave.

Sources:
- Employee Handbook, page 14
- HR Policy, section 3.2

Citations improve:

text
Trust
Verification
Auditability

22. Document Versioning

Documents change.

Store:

text
Document
Version
Effective date
Status

Retrieval should prefer the appropriate active version.

23. Project 3: Multimodal Document Assistant

Build an assistant that can understand:

text
PDF
Image
Table
Chart
Text

Example:

Architecture & Data Flow
User uploads financial report
 |
 v
Document processing
 |
 +--> Text extraction
 |
 +--> Image analysis
 |
 +--> Table extraction
 |
 v
Unified context
 |
 v
Multimodal LLM

24. Multimodal Query

Example:

›"Look at this chart and explain why revenue declined in Q3."

The system may combine:

text
Chart image
+
Report text
+
Relevant metadata

before generating the answer.

25. Multimodal RAG

A multimodal retrieval system can index:

text
Text embeddings
Image embeddings
Audio embeddings

or use unified multimodal representations where supported.

Conceptually:

Architecture & Data Flow
User query
 |
 +--> Text retrieval
 |
 +--> Image retrieval
 |
 +--> Document retrieval
 |
 v
Context fusion
 |
 v
Multimodal model

26. Project 4: Research Agent

Build an agent that can:

text
Search information
Read documents
Extract facts
Compare sources
Calculate values
Generate a report

27. Research Agent Architecture

Architecture & Data Flow
User
 |
 v
Planner
 |
 +--> Search
 |
 +--> Document reader
 |
 +--> Calculator
 |
 +--> Database
 |
 v
Evidence collection
 |
 v
Synthesis
 |
 v
Report

28. LangGraph-Style Workflow

Conceptually:

Architecture & Data Flow
START
 |
 v
Understand request
 |
 v
Plan research
 |
 v
Search
 |
 v
Evaluate evidence
 |
 +---- insufficient ----> Search again
 |
 v
Synthesize
 |
 v
Validate
 |
 v
END

This is better represented as a stateful workflow than an uncontrolled loop.

29. Agent State

Example:

🐍 Python
state = {
 "question": "...",
 "search_results": [],
 "documents": [],
 "facts": [],
 "citations": [],
 "draft": None
}

Each node updates the state.

30. Tool Permissions

The research agent might have:

text
Search: allowed
Calculator: allowed
Database: read-only
Email: not allowed
Payments: not allowed

Least privilege is important.

31. Research Agent Evaluation

Evaluate:

text
Search quality
Source quality
Citation correctness
Fact extraction
Tool selection
Tool arguments
Final answer quality

A fluent answer is not enough.

32. Project 5: AI Customer Support Agent

Build a support assistant that can:

text
Answer FAQs
Look up orders
Check account information
Create support tickets
Escalate to humans

33. Support Agent Architecture

Architecture & Data Flow
Customer
 |
 v
Chat UI
 |
 v
Support API
 |
 v
Agent
 |
 +--> Knowledge base
 |
 +--> Order API
 |
 +--> Ticket system
 |
 +--> Human escalation
 |
 v
Response

34. Tool Calling

Example:

json
{
 "name": "get_order_status",
 "arguments": {
 "order_id": "12345"
 }
}

The backend should validate:

text
Tool name
Arguments
User authorization
Tenant
Rate limits

35. Human Handoff

The agent should escalate when:

text
User requests human support
Issue is high risk
Confidence is low
Policy requires escalation
Repeated failures occur

Workflow:

Architecture & Data Flow
Agent
 |
 v
Escalation decision
 |
 v
Human support

36. Project 6: AI Content Generation Platform

Build a platform for teachers or organizations to generate:

text
Lessons
Quizzes
Flashcards
Summaries
Practice questions
Study guides

37. Content Generation Workflow

Architecture & Data Flow
Teacher
 |
 v
Select subject
 |
 v
Select grade
 |
 v
Select topic
 |
 v
Choose content type
 |
 v
Generate
 |
 v
Validate
 |
 v
Teacher review
 |
 v
Publish

Human review is especially useful before publishing educational content.

38. Structured Generation

A quiz generator might produce:

json
{
 "title": "Fractions Quiz",
 "questions": [
 {
 "question": "What is 1/2 + 1/4?",
 "options": ["1/4", "2/4", "3/4", "4/4"],
 "answer": "3/4",
 "difficulty": "easy"
 }
 ]
}

Validate the response against a schema.

39. Project 7: AI Study Planner

Build a personalized study planner.

Inputs:

text
Subjects
Upcoming exams
Available time
Completed lessons
Weak topics
Target dates

Output:

text
Daily plan
Review schedule
Practice tasks
Revision recommendations

The LLM should generate the plan, but deterministic scheduling logic should enforce hard constraints.

40. Hybrid AI Architecture

A useful pattern:

Architecture & Data Flow
LLM
 |
 v
Proposed plan
 |
 v
Deterministic scheduler
 |
 v
Validated plan

Do not delegate strict business rules to a probabilistic model.

41. Project 8: Voice AI Tutor

Combine:

text
Speech-to-text
+
LLM
+
Text-to-speech

Pipeline:

Architecture & Data Flow
Student voice
 |
 v
Speech recognition
 |
 v
LLM
 |
 v
Response text
 |
 v
Speech synthesis
 |
 v
Student

42. Voice Tutor Considerations

Optimize:

text
Speech recognition latency
LLM TTFT
Text-to-speech latency
Turn-taking
Interruption handling

Streaming becomes especially important.

43. Project 9: Video Learning Assistant

A video assistant can:

text
Summarize lectures
Generate chapters
Extract key concepts
Answer questions about the video
Generate quizzes

Pipeline:

Architecture & Data Flow
Video
 |
 +--> Audio
 | |
 | v
 | Speech-to-text
 |
 +--> Frames
 |
 v
 Vision analysis
 |
 +---------+
 |
 v
 Unified index
 |
 v
 RAG
 |
 v
 LLM

44. Timestamped Retrieval

Store:

text
Video ID
Start time
End time
Transcript
Frame reference
Topic

Then answers can reference:

›Lecture 12:34–14:02

This improves learning usability.

45. Project 10: Sovereign Educational AI

Build a school-local AI deployment.

Architecture:

Architecture & Data Flow
School Network
 |
 v
Local API
 |
 v
Local RAG
 |
 v
Local Open-Weight Model
 |
 v
Student / Teacher Apps

The system should be able to operate without sending sensitive educational data to an external model API.

46. Sovereign Deployment Requirements

Consider:

text
Local model weights
Offline inference
Local vector database
Local authentication
Local logging
Network isolation
Controlled updates
Model provenance

47. Shared Components Across Projects

Instead of rebuilding everything for every application, create reusable services:

text
Authentication service
AI gateway
Model router
RAG service
Vector database service
Evaluation service
Observability service
Usage service

This creates a platform architecture.

48. Reusable AI Gateway

Example:

🐍 Python
class AIGateway:

 def generate(
 self,
 model,
 messages,
 temperature=0.2
 ):
 ...

Applications call:

🐍 Python
gateway.generate(...)

instead of directly depending on a specific provider.

49. Model Provider Abstraction

Conceptually:

🐍 Python
class ModelProvider:
 def generate(self, messages):
 raise NotImplementedError

Implementations:

text
LocalProvider
OpenAICompatibleProvider
CloudProvider
MockProvider

This makes testing and model migration easier.

50. RAG Service Abstraction

🐍 Python
class RetrievalService:

 def search(
 self,
 query,
 tenant_id,
 filters=None
 ):
 ...

Applications can share:

text
Chunking
Embedding
Retrieval
Reranking
Citation

logic.

51. Evaluation Service

A reusable evaluation service can run:

text
Golden datasets
Regression tests
Safety tests
Groundedness checks
Structured-output checks

Example:

Architecture & Data Flow
Model version
 |
 v
Evaluation suite
 |
 v
Score
 |
 +--> pass
 |
 +--> fail

52. Configuration Management

Avoid hardcoding:

🐍 Python
MODEL = "some-model"

Use configuration:

text
MODEL_NAME
MODEL_ENDPOINT
TEMPERATURE
MAX_TOKENS
VECTOR_DB
EMBEDDING_MODEL

This makes deployment environments easier to manage.

53. Secrets Management

Never put API keys directly into code.

Bad:

🐍 Python
API_KEY = "secret-value"

Prefer:

text
Environment variables
Secret manager
Platform-managed credentials

54. Database Architecture

A production application may use:

text
PostgreSQL
+
Vector database
+
Object storage
+
Cache

Example:

Architecture & Data Flow
PostgreSQL
 -> users
 -> courses
 -> permissions

Vector DB
 -> embeddings

Object Storage
 -> PDFs
 -> images
 -> videos

Redis
 -> sessions
 -> cache

55. API Layer

FastAPI can expose:

text
Authentication
Courses
Lessons
Chat
RAG
Files
AI generation
Evaluation
Usage

Example:

🐍 Python
from fastapi import FastAPI

app = FastAPI()

@app.get("/health")
def health():
 return {"status": "ok"}

56. Streaming API

Conceptually:

🐍 Python
from fastapi.responses import StreamingResponse

@app.post("/chat")
def chat():
 return StreamingResponse(
 generate_tokens(),
 media_type="text/event-stream"
 )

The exact production implementation should also handle:

text
Disconnects
Timeouts
Cancellation
Authentication
Backpressure

57. Authentication

Typical flow:

Architecture & Data Flow
Login
 |
 v
Access token
 |
 v
API request
 |
 v
Validate token
 |
 v
Identify user
 |
 v
Authorize resource

Authentication answers:

›Who are you?

Authorization answers:

›What are you allowed to access?

58. Authorization

For a course:

Architecture & Data Flow
Student A -> enrolled -> allowed
Student B -> not enrolled -> denied
Teacher A -> owns course -> allowed

Authorization must happen before retrieving protected data.

59. Multi-Tenant Architecture

Architecture & Data Flow
Tenant A
 |
 +--> Users
 +--> Courses
 +--> Documents
 +--> Vector data

Tenant B
 |
 +--> Users
 +--> Courses
 +--> Documents
 +--> Vector data

Every data-access layer should preserve tenant boundaries.

60. Observability

Track:

text
Request ID
User ID
Tenant ID
Model
Prompt version
Input tokens
Output tokens
Latency
Retriever latency
Tool calls
Errors

Sensitive content should be handled according to privacy policy.

61. End-to-End Trace

Example:

Architecture & Data Flow
Request
 |
 +--> Auth: 10ms
 |
 +--> Retrieval: 70ms
 | |
 | +--> Vector DB: 45ms
 |
 +--> LLM: 900ms
 |
 +--> Validation: 5ms
 |
 v
Response

This makes performance debugging much easier.

62. Evaluation Dataset

Create a representative dataset:

text
Question
Expected behavior
Reference answer
Relevant documents
Difficulty
Category
Safety label

Example:

json
{
 "question": "Explain Newton's second law.",
 "expected": "Correct, age-appropriate explanation.",
 "grade": "8"
}

63. Regression Testing

Every major change should run:

text
Prompt change
Model change
RAG change
Retriever change
Fine-tuning change

against the evaluation suite.

64. Human Evaluation

Automated metrics are useful but not sufficient.

Humans can evaluate:

text
Clarity
Helpfulness
Pedagogical quality
Tone
Correctness
Citation quality

Use structured rubrics.

65. Red Teaming

Test intentionally difficult inputs:

text
Prompt injection
Data extraction
Unsafe requests
Instruction conflicts
Cross-tenant retrieval
Tool misuse
Malformed inputs

The goal is to discover weaknesses before users do.

66. Cost Monitoring

For every request, estimate:

text
Input token cost
Output token cost
Embedding cost
Retrieval cost
Multimodal processing cost
Infrastructure cost

Then calculate:

text
Cost per request
Cost per active user
Cost per course

67. Performance Optimization

Optimize in this order:

Architecture & Data Flow
Measure
 |
 v
Find bottleneck
 |
 v
Optimize
 |
 v
Measure again

Possible improvements:

text
Caching
Smaller model
Quantization
Prompt compression
Context reduction
Better retrieval
Batching
Model routing

68. Production Deployment

A basic deployment pipeline:

Architecture & Data Flow
Git repository
 |
 v
Tests
 |
 v
Evaluation
 |
 v
Build container
 |
 v
Staging
 |
 v
Canary
 |
 v
Production

A model or prompt change should not bypass evaluation.

69. CI/CD Pipeline

Example:

Architecture & Data Flow
Pull Request
 |
 v
Unit tests
 |
 v
Integration tests
 |
 v
AI evaluation
 |
 v
Security checks
 |
 v
Build
 |
 v
Deploy staging

70. Project Selection Guide

Choose a project based on your goal:

GoalRecommended project
Learn RAGDocument Intelligence
Learn agentsResearch Agent
Learn toolsCustomer Support Agent
Learn multimodal AIMultimodal Document Assistant
Learn voice AIVoice Tutor
Learn video AIVideo Learning Assistant
Build an educational productAI Tutor
Learn sovereigntySovereign Educational AI
Learn production architectureAI Gateway + LLMOps platform

71. Recommended Portfolio Sequence

For a strong portfolio:

Architecture & Data Flow
Project 1
Educational AI Tutor
 |
 v
Project 2
Document RAG
 |
 v
Project 3
Agentic Research Assistant
 |
 v
Project 4
Multimodal Assistant
 |
 v
Project 5
Production AI Gateway
 |
 v
Project 6
Sovereign AI Deployment

Each project builds on the previous one.

72. Portfolio Project Requirements

Every serious project should document:

text
Problem
Users
Requirements
Architecture
Model
Prompt strategy
RAG strategy
Tools
Evaluation
Security
Latency
Cost
Deployment
Limitations
Future improvements

This demonstrates engineering maturity.

73. Architecture Documentation

Create:

text
README.md
architecture.md
API documentation
evaluation.md
deployment.md

Also include diagrams where useful.

74. Repository Structure

A practical project structure:

Architecture & Data Flow
genai-project/
|
+-- app/
| +-- api/
| +-- models/
| +-- services/
| +-- retrieval/
| +-- agents/
| +-- evaluation/
|
+-- data/
|
+-- tests/
|
+-- prompts/
|
+-- configs/
|
+-- scripts/
|
+-- Dockerfile
+-- requirements.txt
+-- README.md

75. Testing Strategy

Use multiple testing layers:

text
Unit tests
Integration tests
API tests
Retrieval tests
Prompt tests
Model evaluation
Security tests
Load tests

LLM applications require more than traditional unit tests.

76. Mocking Models

During development, use a mock model where possible.

Example:

🐍 Python
class MockModel:

 def generate(self, messages):
 return "mock response"

This makes application tests:

text
Fast
Cheap
Deterministic

77. Deterministic Business Logic

Keep strict rules outside the LLM.

Examples:

text
User permissions
Pricing
Course enrollment
Exam deadlines
Payment status
Database updates

Use the LLM for:

text
Language
Reasoning assistance
Summarization
Classification
Content generation

78. AI + Traditional Software

A powerful architecture is:

text
Traditional software
 +
Probabilistic AI

Use deterministic code for:

text
Rules
Security
Transactions
Validation
State

Use AI for:

text
Language
Interpretation
Generation
Semantic matching

79. Production Readiness Checklist

Architecture#

text
Clear services
API boundaries
Data boundaries
Model abstraction

AI#

text
Model evaluated
Prompt versioned
RAG evaluated
Tools validated

Security#

text
Authentication
Authorization
Tenant isolation
Secret management
Prompt injection defenses

Operations#

text
Logging
Metrics
Tracing
Alerts
Cost monitoring

Reliability#

text
Timeouts
Retries
Fallbacks
Rate limits
Circuit breakers

Deployment#

text
CI/CD
Staging
Canary
Rollback

80. Final Capstone: Educational AI Platform

The final capstone can combine everything.

Architecture & Data Flow
 EDUCATIONAL PLATFORM
 |
 +------------------+------------------+
 | | |
 v v v
 Students Teachers Admins
 | | |
 +------------------+------------------+
 |
 v
 API Gateway
 |
 v
 Authentication
 |
 v
 AI Gateway
 |
 +------------------+------------------+
 | | |
 v v v
 Cache Router Safety
 |
 +----------------+----------------+
 | | |
 v v v
 Small LLM Medium LLM Large LLM
 | | |
 +----------------+----------------+
 |
 +------------------+------------------+
 | | |
 v v v
 RAG Tools Multimodal
 | | |
 v v v
 Course Data Platform APIs Images/Audio
 | | |
 +------------------+------------------+
 |
 v
 Output Validation
 |
 v
 Student / Teacher
 |
 v
 Evaluation / Observability

81. Capstone Features

The educational platform can eventually support:

text
AI Tutor
AI Quiz Generator
AI Lesson Generator
AI Study Planner
AI Homework Assistant
Document Q&A
Voice Tutor
Video Learning Assistant
Teacher Copilot
Personalized Practice

82. Capstone AI Tutor Modes

Possible modes:

text
Explain
Hint
Practice
Quiz
Review
Summarize
Ask

Each mode can use a different:

text
Prompt
Model
Temperature
Tool set
Evaluation rubric

83. Capstone RAG Architecture

Architecture & Data Flow
Teacher Content
 |
 +--> Lessons
 +--> PDFs
 +--> Markdown
 +--> Images
 +--> Videos
 |
 v
Content Processing
 |
 v
Chunking
 |
 v
Embeddings
 |
 v
Vector DB
 |
 v
Tenant + Course Filters
 |
 v
Retriever
 |
 v
Reranker
 |
 v
LLM

84. Capstone Personalization

Student-specific information may include:

text
Course
Current lesson
Practice history
Weak topics
Difficulty preference

Use only the context required for the current task.

85. Capstone Evaluation

Measure:

text
Correctness
Groundedness
Age/level appropriateness
Pedagogical usefulness
Hint quality
Question quality
Safety
Latency
Cost
Student feedback

86. Capstone Production Stack

One possible stack:

Architecture & Data Flow
Frontend
 -> React / Next.js

Backend
 -> FastAPI

Database
 -> PostgreSQL

Vector Search
 -> FAISS / pgvector / vector database

Cache
 -> Redis

Models
 -> Local or approved model APIs

Inference
 -> vLLM / llama.cpp / appropriate runtime

Storage
 -> Object storage

Observability
 -> Logs + metrics + traces

Deployment
 -> Docker / Kubernetes where justified

The exact technologies can be changed without changing the architecture.

87. Capstone Development Phases

Phase 1#

Build:

text
Authentication
Courses
Lessons
Basic chat

Phase 2#

Add:

text
RAG
Citations
Teacher content

Phase 3#

Add:

text
Hints
Quiz generation
Personalization

Phase 4#

Add:

text
Multimodal
Voice
Video

Phase 5#

Add:

text
Evaluation
Observability
Cost monitoring

Phase 6#

Add:

text
Production scaling
Model routing
Sovereign deployment

88. Final Mental Model

You should now be able to connect:

Architecture & Data Flow
Foundation Models
 |
 v
Prompting
 |
 v
RAG
 |
 v
Agents
 |
 v
Multimodal AI
 |
 v
Fine-Tuning
 |
 v
Open / Sovereign Models
 |
 v
LLMOps
 |
 v
Production Applications

The real skill is not knowing each technology independently.

It is knowing:

text
When to use it
Why to use it
How to combine it
How to evaluate it
How to secure it
How to operate it

89. Key Takeaways

  1. Real GenAI systems combine multiple technologies.
  2. Start with the user problem, not the model.
  3. RAG is useful for external or changing knowledge.
  4. Agents are useful when workflows require tools and dynamic decisions.
  5. Multimodal systems can combine specialized modality pipelines.
  6. Fine-tuning is useful for persistent behavioral adaptation.
  7. Model gateways abstract inference providers.
  8. Production systems need authentication and authorization.
  9. Multi-tenant retrieval must enforce strict data isolation.
  10. Structured outputs should be validated.
  11. Deterministic business rules should remain outside the LLM.
  12. AI quality requires dedicated evaluation.
  13. Observability should cover infrastructure and AI behavior.
  14. Caching can reduce cost and latency.
  15. Model routing can optimize quality, latency, and cost.
  16. Streaming improves interactive user experience.
  17. Educational AI should optimize for learning, not only answer correctness.
  18. Teacher-controlled content can provide a trusted knowledge boundary.
  19. Voice and video applications require specialized latency-aware pipelines.
  20. Sovereign deployments require control over models, data, infrastructure, and operations.
  21. Portfolio projects should document architecture, evaluation, security, cost, and limitations.
  22. Production GenAI is a combination of traditional software engineering and probabilistic AI.
  23. The strongest systems use deterministic software for rules and AI for language and semantic tasks.
  24. End-to-end engineering is the bridge between learning GenAI concepts and building useful products.

90. Knowledge Check

Question 1#

Why should you define the problem before selecting a model?

Question 2#

What components typically appear in a production GenAI architecture?

Question 3#

How would you build an educational AI tutor using RAG?

Question 4#

Why is tenant-aware retrieval important?

Question 5#

When should an AI tutor provide a hint instead of a full answer?

Question 6#

How can a research agent use multiple tools?

Question 7#

Why should deterministic business rules remain outside the LLM?

Question 8#

What metrics would you use to evaluate an AI tutor?

Question 9#

How would you build a voice tutor?

Question 10#

How would you process an educational video?

Question 11#

What role does an AI gateway play?

Question 12#

Why are prompt and model versioning important?

Question 13#

What should a production GenAI CI/CD pipeline test?

Question 14#

How can model routing reduce cost?

Question 15#

What makes an AI system suitable for sovereign deployment?

91. Final Project Challenge

Build a complete AI Learning Assistant with:

text
1. Student authentication
2. Teacher authentication
3. Course management
4. Lesson management
5. Document upload
6. RAG
7. AI tutor
8. Hint mode
9. Quiz generation
10. Structured outputs
11. Conversation history
12. Usage limits
13. Evaluation dataset
14. Observability
15. Cost tracking
16. Multimodal input
17. Model routing
18. Production API

Start small.

A strong first version can be:

text
Authentication
+
Courses
+
Lessons
+
RAG
+
AI Tutor

Then progressively add the remaining components.

That approach is much more realistic than attempting the entire platform at once.

Knowledge Checkpoint

End-to-End GenAI Projects Checkpoint

Q1.In a production Financial Report Analyst GenAI project, why is chunking financial tables by semantic table rows/Markdown tables critical?
AFixed-character chunking slices through table rows, splitting numerical data from corresponding headers and destroying numerical accuracy in RAG.
BFinancial tables cannot be stored in vector databases.
CMarkdown tables double the size of vector embeddings.
DTables can only be read by OCR models.
Q2.What is the recommended pattern for implementing conversational memory in production multi-turn chatbots?
AMaintaining a rolling window of recent message turns combined with periodic asynchronous LLM conversation summarization for long-term context.
BAppending every single past message verbatim forever until the context window crashes.
CStoring only the user's last message.
DWriting conversation history into a static text file on the user's desktop.
Q3.How should streaming GenAI backends handle sudden client disconnections gracefully?
AListen to HTTP request cancellation signals / client disconnect events, immediately aborting the underlying upstream LLM generation to prevent wasted token costs.
BContinue generating all tokens in the background regardless of disconnection.
CCrash the web server.
DCharge the user double.
Track Your Learning

Finished studying this notebook?

Mark this guide as completed to update your course progress roadmap.

Previous Lesson
Future & Research AI Architectures
Next Lesson
Full Generative AI Capstone