DeutschLernen/docs/features/story-integration.md

Feature: Story Integration

Status: 🚀 In Progress
📊 Current Progress: Phase 1-5 ✅ Complete (Database & Models, Backend Services, Unit Tests, AI Integration, Audio Generation + Migration), Phase 6 Next (Frontend Integration) Priority: High
Complexity: High
Estimate: 8-12 hours
Assignee: -
Created: May 31, 2025
Target Completion: -
PR: -
Related Features: Infrastructure Setup, Lesson Management, AI Services (Mistral, TTS)

---

📌 Overview

Purpose

Implement a continuous story narrative that unlocks segment by segment as users complete lessons, with AI-generated content tailored to each level and lesson's vocabulary.

User Story

As a learner, I want to follow a continuous story that uses the vocabulary and grammar I'm learning, so that I can see the language in context and stay motivated through narrative progression.

Acceptance Criteria

• Each level has a continuous story (e.g., A1: "A Week in the Life of Anna")
• Story is divided into segments, one per lesson
• Story segments unlock after completing the associated lesson's quiz
• Story content uses only vocabulary and grammar from completed lessons
• Each story segment has associated audio (Coqui TTS)
• Users can click on words to see translations
• Story progress is tracked separately from lesson progress

---

📋 Requirements

Functional Requirements

ID	Requirement	Priority
FR-001	Create/Read/Update/Delete story segments (Admin)	High
FR-002	Associate story segments with levels and lessons	High
FR-003	Generate story content using Mistral-Medium AI	High
FR-004	Generate audio for story segments using Coqui TTS	High
FR-005	Sequential story unlocking based on lesson completion	High
FR-006	Display story with clickable words for translations	High
FR-007	Track story progress per user	Medium
FR-008	Support multiple stories per level (optional)	Low

Non-Functional Requirements

• Performance: Story segment loading < 300ms
• Storage: Audio files ~100-500KB per segment
• Security: Only admins can create/edit story segments
• Data Integrity: Story-level-lesson associations must be valid

---

🏗️ Technical Design

Components Involved

• Backend: StoryController, StoryService, StoryGenerationService
• Database: StorySegments table
• Models: StorySegment, StorySegmentDto, StoryGenerationRequest
• External: Mistral-Medium API, Coqui TTS
• Frontend: StoryPage, StoryTab, StorySegment, StoryPlayer components

Data Flow

1. Admin triggers story generation for a level
2. Backend calls Mistral-Medium with vocabulary from all lessons in level
3. Mistral returns story text divided into segments
4. For each segment:
   a. Store in StorySegments table
   b. Associate with level and lesson
   c. Generate audio using Coqui TTS
   d. Save audio file and update AudioUrl
5. User completes lesson quiz (80%+)
6. Backend unlocks next story segment for user
7. User reads/listens to story segment
8. User can click on words to see translations

API Endpoints

Endpoint	Method	Description	Auth Required
/api/story/levels	GET	List levels with stories	Yes
/api/story/{levelId}/segments	GET	List story segments for a level	Yes
/api/story/segments/{id}	GET	Get specific story segment	Yes
/api/story/segments	POST	Create/update story segment (Admin)	Yes
/api/story/segments/{id}	DELETE	Delete story segment (Admin)	Yes
/api/story/generate	POST	Generate story for level (Admin)	Yes
/api/story/{levelId}/progress	GET	Get user's story progress for level	Yes
/api/story/segments/{id}/audio	GET	Get audio file for segment	Yes

Database Schema (from application-plan.md)

CREATE TABLE StorySegments (
    Id SERIAL PRIMARY KEY,
    LevelId INT REFERENCES Levels(Id) ON DELETE CASCADE,
    LessonId INT REFERENCES Lessons(Id) ON DELETE SET NULL,
    Content TEXT NOT NULL,
    AudioUrl VARCHAR(255),
    Order INT NOT NULL,
    UNIQUE(LevelId, Order)
);

Story Generation Prompt

Write a 2–3 paragraph story about daily life for a German {level} learner. 
Use only {level} vocabulary and grammar (present tense, basic nouns/verbs). 
Include the following words: {vocabularyList}. 
Theme: "{theme}".
Divide the story into {segmentCount} segments, one for each lesson.

Example Story Segment (from application-plan.md)

Id: 1
LevelId: 1 (A1)
LessonId: 1
Content: "Anna steht jeden Morgen um 7 Uhr auf. Sie geht ins Badezimmer und wäscht sich das Gesicht. Dann frühstückt sie in der Küche. Sie isst ein Brot und trinkt einen Kaffee."
AudioUrl: "/audio/story/1.wav"
Order: 1

---

🚀 Implementation Plan

Phase 1: Database & Models (2 hours)

• Create StorySegment entity (Domain/Entities/StorySegment.cs)
• Create StoryProgress entity (Domain/Entities/StoryProgress.cs)
• Create StorySegmentDto (Application/DTOs/StorySegmentDto.cs)
• Create StoryGenerationRequestDto
• Create IStoryRepository interface (Domain/Interfaces/IStoryRepository.cs)
• Create IStoryProgressRepository interface (Domain/Interfaces/IStoryProgressRepository.cs)
• Create StoryRepository implementation (Infrastructure/Data/Repositories/StoryRepository.cs)
• Create StoryProgressRepository implementation (Infrastructure/Data/Repositories/StoryProgressRepository.cs)
• Add DbSets to AppDbContext (StorySegments, StoryProgress)
• Add entity configurations to AppDbContext
• Add navigation properties to Level, Lesson, and User entities
• Create and apply migration for StorySegments and StoryProgress tables

Phase 2: Backend Services (2-3 hours)

• Create StoryService with CRUD operations (Application/Services/StoryService.cs)
• Create StoryGenerationService for AI integration (Application/Services/StoryGenerationService.cs)
• Implement story segment generation using MistralService
• Implement audio generation using ITtsService
• Create StoryUnlockService for progress management (Application/Services/StoryUnlockService.cs)
• Create Presentation/Controllers/StoryController.cs (12 endpoints)

Phase 3: Unit Tests (2-3 hours)

• Write unit tests for StoryService (Tests/Unit/Application/Services/StoryServiceTests.cs)
• Write unit tests for StoryGenerationService (Tests/Unit/Application/Services/StoryGenerationServiceTests.cs)
• Write unit tests for StoryUnlockService (Tests/Unit/Application/Services/StoryUnlockServiceTests.cs)
• Write unit tests for StoryRepository (Tests/Unit/Infrastructure/Data/Repositories/StoryRepositoryTests.cs)
• Write unit tests for StoryProgressRepository (Tests/Unit/Infrastructure/Data/Repositories/StoryProgressRepositoryTests.cs)
• All 324 tests pass

Phase 4: AI Integration (2-3 hours) - COMPLETE

• Configure Mistral-Medium API client (MistralConfig with retry, rate limiting, circuit breaker)
• Create prompt templates for each level (A1-C1 with CEFR-specific requirements)
• Implement vocabulary extraction from lessons (ExtractVocabularyFromLessons)
• Implement story text segmentation (SplitStoryIntoSegments, AdjustSegmentCount)
• Handle AI API errors gracefully (AiServiceException with error codes)
• Add retry logic for failed generations (ExecuteWithRetryAsync with exponential backoff)

Phase 5: Audio Generation (2 hours) - COMPLETE

• Integrate with Coqui TTS service (ITtsService, TtsService with CoquiConfig)
• Generate audio for each story segment (GenerateAudioAsync in StoryGenerationService)
• Store audio files with consistent naming (wwwroot/audio/story/level{id}-segment{order}.wav)
• Update StorySegment.AudioUrl after generation (segment.UpdateAudioUrl)
• Add audio file serving endpoint (UseStaticFiles + GET segments/{id}/audio)

Phase 5: Frontend Integration (2-3 hours)

• Create StoryPage component
• Create StoryTab component
• Create StorySegment component
• Create StoryPlayer component with audio
• Implement word click-to-translate functionality
• Add story progress tracking
• Integrate with LessonPage

Phase 6: User Progress (1-2 hours)

• Track which story segments user has unlocked
• Update unlock status when lesson is completed
• Display locked segments as "coming soon"
• Show progress through story

Milestones

Milestone	Date	Status
Database & Models	June 13, 2025	✅
Backend Services	June 13, 2025	✅
Unit Tests	June 13, 2025	✅
AI Integration	June 13, 2025	✅
Audio Generation & Migration	June 13, 2025	✅
Frontend Integration	-	⏳
User Progress	-	⏳

---

✅ Tasks

Backend

• Create Domain/Entities/StorySegment.cs
• Create Domain/Entities/StoryProgress.cs
• Create Application/DTOs/StorySegmentDto.cs
• Create Application/DTOs/StoryGenerationRequestDto.cs
• Create Domain/Interfaces/IStoryRepository.cs
• Create Domain/Interfaces/IStoryProgressRepository.cs
• Create Infrastructure/Data/Repositories/StoryRepository.cs
• Create Infrastructure/Data/Repositories/StoryProgressRepository.cs
• Add DbSets and configurations to AppDbContext
• Update Level, Lesson, and User entities with navigation properties
• Create Application/Services/StoryService.cs
• Create Application/Services/StoryGenerationService.cs (uses IMistralService, ITtsService)
• Create Application/Services/StoryUnlockService.cs (handles lesson completion → story unlocking)
• Create Presentation/Controllers/StoryController.cs (12 endpoints)
• Register services in Program.cs (IStoryRepository, IStoryProgressRepository, StoryService, StoryGenerationService, StoryUnlockService)
• Write unit tests (324 tests: StoryService, StoryGenerationService, StoryUnlockService, StoryRepository, StoryProgressRepository)
• Write integration tests

Database

• Add DbSets and configurations to AppDbContext
• Add navigation properties to Level, Lesson, User entities
• Create migration for StorySegments and StoryProgress tables
• Add foreign keys to Levels and Lessons
• Add indexes for LevelId, LessonId, Order
• Apply migration

AI Integration

• Set up Mistral API client with configuration
• Create prompt templates for each CEFR level
• Implement vocabulary extraction from lessons
• Implement story segmentation logic
• Add error handling for Mistral API
• Add rate limiting/caching for Mistral calls

Audio Generation

• Extend Coqui TTS service for story segments
• Implement batch audio generation
• Create audio file storage structure
• Add audio serving endpoint
• Implement audio cleanup mechanism

Frontend

• Create pages/StoryPage.tsx
• Create components/StoryTab.tsx
• Create components/StorySegment.tsx
• Create components/StoryPlayer.tsx
• Create hooks/useStory.ts
• Create hooks/useStoryAudio.ts
• Implement word translation on click
• Add story progress indicator
• Add navigation between segments

User Progress

• Extend UserProgress or create StoryProgress table
• Implement unlock logic when lesson completed
• Create endpoint to get user's story progress
• Add locked segment UI state

---

🔗 Dependencies

Feature Dependencies

• Infrastructure Setup - Required
• Lesson Management - Required (story segments associated with lessons/levels)
• AI Services - Required (Mistral for generation, Coqui for audio)

Technical Dependencies

• Mistral-Medium API access
• Coqui TTS Python library
• Entity Framework Core

Blockers

• Infrastructure Setup must be complete
• Lesson Management must be complete
• AI Services must be configured
• Mistral API key required

---

✅ Definition of Done

General Criteria (All Features)

• All acceptance criteria met and verified
• All tasks in this document completed
• Code follows Clean Architecture principles
• Code reviewed and approved by at least 1 team member
• All tests passing (unit, integration)
• Documentation updated (README, AGENTS.md if applicable)
• Feature works in development environment
• Feature deployed to staging environment
• Performance meets defined targets
• Security review completed
• No critical bugs or blockers

Story-Specific Criteria

• Story segments can be created, read, updated, and deleted
• Stories are associated with levels and lessons
• Story content is generated using Mistral-Medium
• Story audio is generated using Coqui TTS
• Story segments unlock sequentially with lesson completion
• Users can view unlocked story segments
• Users can listen to story audio
• Users can click on words to see translations
• Story progress is tracked per user

---

🧪 Testing Strategy

Testing Approach

Test Type	Coverage	Tools	Responsibility
Unit Tests	80%+ code coverage	MsTest, Moq	Backend Dev
Integration Tests	All service interactions	MsTest, TestContainers	Backend Dev
API Tests	All endpoints	MsTest, HttpClient	Backend Dev
Frontend Unit Tests	Component logic	Vitest	Frontend Dev
Frontend Integration	Service integration	Vitest	Frontend Dev
E2E Tests	Critical user journeys	Playwright	QA/Dev
Manual Testing	Exploratory, edge cases	BrowserStack	QA

Story-Specific Tests

Backend Tests

• Create story segment → success
• Create story segment with invalid level → error
• Get story segments by level → returns correct list
• Get story segment by ID → returns correct segment
• Update story segment → success
• Delete story segment → success
• Generate story with Mistral → returns valid story text
• Generate story with vocabulary list → includes all words
• Generate audio for story segment → creates audio file
• Get user's story progress → returns correct status

AI Tests

• Mistral generates valid German story for A1 level
• Mistral generates story with all requested vocabulary
• Mistral handles different CEFR levels appropriately
• Coqui TTS generates clear audio for story text
• Coqui TTS handles long story segments
• Coqui TTS handles German special characters (umlauts, ß)

Frontend Tests

• Story page loads and displays segments
• Audio plays for story segment
• Word click shows translation
• Navigation between segments works
• Locked segments are displayed as "coming soon"
• Story progress is displayed correctly

---

📝 Notes & Decisions

Decisions Made

Date	Decision	Rationale
May 31, 2025	One story per level for MVP	Simpler implementation, can expand later
May 31, 2025	Segment per lesson	Natural progression, matches lesson structure
May 31, 2025	Generate audio for all segments	Essential for accessibility and pronunciation
May 31, 2025	Use Mistral-Medium	High-quality, cost-effective for this use case
May 31, 2025	Click-to-translate words	Enhances learning without disrupting flow

Technical Notes

• StorySegments.LessonId can be NULL for intro/conclusion segments
• Order field ensures segments appear in correct sequence
• UNIQUE(LevelId, Order) prevents duplicate ordering
• Content should be stored as plain text, formatted on frontend
• Audio files should be named: /audio/story/{levelId}-{order}.wav

Prompt Engineering

Different prompts for different levels:

• A1: Simple present tense, basic vocabulary, short sentences
• A2: Present, past, future tense, expanded vocabulary
• B1: Complex sentences, subordinate clauses, broader topics
• B2/C1: Advanced grammar, idiomatic expressions, nuanced topics

Gotchas

• ⚠️ Mistral API has rate limits - implement caching
• ⚠️ AI generation may produce non-A1 vocabulary - need validation
• ⚠️ Long stories may exceed token limits - need segmentation
• ⚠️ Audio generation for long segments may be slow
• ⚠️ Need to handle cases where vocabulary list is empty

Example Prompts by Level

A1 Prompt:

Write a 2-3 paragraph story about daily life in Germany for an A1 learner.
Use only A1 vocabulary and simple present tense sentences.
Include these words: {vocabularyList}
Theme: {theme}

B1 Prompt:

Write a 3-4 paragraph story about {theme} for a B1 German learner.
Use B1-level vocabulary and grammar including present, past, and future tenses.
Include these words: {vocabularyList}
Make the story engaging and suitable for adult learners.

---

📊 Progress History

Date	Status Change	Notes
May 31, 2025	Created	Initial plan based on application-plan.md
June 13, 2025	Phase 1-2 Complete	Database & Models, Backend Services implemented
June 13, 2025	Phase 3 Complete	All unit tests written and passing (324 tests)
June 13, 2025	Phase 4 Complete	Enhanced prompts with CEFR-level-specific requirements for story generation
June 13, 2025	Phase 5 Complete	Audio generation with Coqui TTS, static file serving, and endpoints

---

📎 Related Files & Links

• Architecture: Backend Structure
• Architecture: Application Plan
• Database Schema: Initial Database Schema
• Feature: Lesson Management
• Feature: AI Services
• Reference: Mistral AI API
• Reference: Coqui TTS

---

Feature created from application-plan.md