DeutschLernen/docs/features/story-integration.md

# Feature: Story Integration

> **Status**: ⏳ Planned  
> **Priority**: High  
> **Complexity**: High  
> **Estimate**: 8-12 hours  
> **Assignee**: -  
> **Created**: May 31, 2025  
> **Target Completion**: -  
> **PR**: -  
> **Related Features**: Infrastructure Setup, Lesson Management, AI Services (Mistral, TTS)

---

## 📌 Overview

### Purpose
Implement a continuous story narrative that unlocks segment by segment as users complete lessons, with AI-generated content tailored to each level and lesson's vocabulary.

### User Story
As a learner, I want to follow a continuous story that uses the vocabulary and grammar I'm learning, so that I can see the language in context and stay motivated through narrative progression.

### Acceptance Criteria
- [ ] Each level has a continuous story (e.g., A1: "A Week in the Life of Anna")
- [ ] Story is divided into segments, one per lesson
- [ ] Story segments unlock after completing the associated lesson's quiz
- [ ] Story content uses only vocabulary and grammar from completed lessons
- [ ] Each story segment has associated audio (Coqui TTS)
- [ ] Users can click on words to see translations
- [ ] Story progress is tracked separately from lesson progress

---

## 📋 Requirements

### Functional Requirements
| ID | Requirement | Priority |
|----|-------------|----------|
| FR-001 | Create/Read/Update/Delete story segments (Admin) | High |
| FR-002 | Associate story segments with levels and lessons | High |
| FR-003 | Generate story content using Mistral-Medium AI | High |
| FR-004 | Generate audio for story segments using Coqui TTS | High |
| FR-005 | Sequential story unlocking based on lesson completion | High |
| FR-006 | Display story with clickable words for translations | High |
| FR-007 | Track story progress per user | Medium |
| FR-008 | Support multiple stories per level (optional) | Low |

### Non-Functional Requirements
- Performance: Story segment loading < 300ms
- Storage: Audio files ~100-500KB per segment
- Security: Only admins can create/edit story segments
- Data Integrity: Story-level-lesson associations must be valid

---

## 🏗️ Technical Design

### Components Involved
- **Backend**: StoryController, StoryService, StoryGenerationService
- **Database**: StorySegments table
- **Models**: StorySegment, StorySegmentDto, StoryGenerationRequest
- **External**: Mistral-Medium API, Coqui TTS
- **Frontend**: StoryPage, StoryTab, StorySegment, StoryPlayer components

### Data Flow
```
1. Admin triggers story generation for a level
2. Backend calls Mistral-Medium with vocabulary from all lessons in level
3. Mistral returns story text divided into segments
4. For each segment:
   a. Store in StorySegments table
   b. Associate with level and lesson
   c. Generate audio using Coqui TTS
   d. Save audio file and update AudioUrl
5. User completes lesson quiz (80%+)
6. Backend unlocks next story segment for user
7. User reads/listens to story segment
8. User can click on words to see translations
```

### API Endpoints
| Endpoint | Method | Description | Auth Required |
|----------|--------|-------------|----------------|
| `/api/story/levels` | GET | List levels with stories | Yes |
| `/api/story/{levelId}/segments` | GET | List story segments for a level | Yes |
| `/api/story/segments/{id}` | GET | Get specific story segment | Yes |
| `/api/story/segments` | POST | Create/update story segment (Admin) | Yes |
| `/api/story/segments/{id}` | DELETE | Delete story segment (Admin) | Yes |
| `/api/story/generate` | POST | Generate story for level (Admin) | Yes |
| `/api/story/{levelId}/progress` | GET | Get user's story progress for level | Yes |
| `/api/story/segments/{id}/audio` | GET | Get audio file for segment | Yes |

### Database Schema (from application-plan.md)
```sql
CREATE TABLE StorySegments (
    Id SERIAL PRIMARY KEY,
    LevelId INT REFERENCES Levels(Id) ON DELETE CASCADE,
    LessonId INT REFERENCES Lessons(Id) ON DELETE SET NULL,
    Content TEXT NOT NULL,
    AudioUrl VARCHAR(255),
    Order INT NOT NULL,
    UNIQUE(LevelId, Order)
);
```

### Story Generation Prompt
```
Write a 2–3 paragraph story about daily life for a German {level} learner. 
Use only {level} vocabulary and grammar (present tense, basic nouns/verbs). 
Include the following words: {vocabularyList}. 
Theme: "{theme}".
Divide the story into {segmentCount} segments, one for each lesson.
```

### Example Story Segment (from application-plan.md)
```
Id: 1
LevelId: 1 (A1)
LessonId: 1
Content: "Anna steht jeden Morgen um 7 Uhr auf. Sie geht ins Badezimmer und wäscht sich das Gesicht. Dann frühstückt sie in der Küche. Sie isst ein Brot und trinkt einen Kaffee."
AudioUrl: "/audio/story/1.wav"
Order: 1
```

---

## 🚀 Implementation Plan

### Phase 1: Database & Models (2 hours)
- [ ] Create StorySegment entity
- [ ] Create StorySegmentDto
- [ ] Create StoryGenerationRequest DTO
- [ ] Create IStoryRepository interface
- [ ] Create StoryRepository implementation
- [ ] Create migration for StorySegments table
- [ ] Add relationships to Level and Lesson entities

### Phase 2: Backend Services (2-3 hours)
- [ ] Create StoryService with CRUD operations
- [ ] Create StoryGenerationService for AI integration
- [ ] Implement Mistral-Medium API client
- [ ] Implement segment generation logic
- [ ] Create story segment ordering logic
- [ ] Write unit tests for services

### Phase 3: AI Integration (2-3 hours)
- [ ] Configure Mistral-Medium API client
- [ ] Create prompt templates for each level
- [ ] Implement vocabulary extraction from lessons
- [ ] Implement story text segmentation
- [ ] Handle AI API errors gracefully
- [ ] Add retry logic for failed generations

### Phase 4: Audio Generation (2 hours)
- [ ] Integrate with Coqui TTS service
- [ ] Generate audio for each story segment
- [ ] Store audio files with consistent naming
- [ ] Update StorySegment.AudioUrl after generation
- [ ] Add audio file serving endpoint

### Phase 5: Frontend Integration (2-3 hours)
- [ ] Create StoryPage component
- [ ] Create StoryTab component
- [ ] Create StorySegment component
- [ ] Create StoryPlayer component with audio
- [ ] Implement word click-to-translate functionality
- [ ] Add story progress tracking
- [ ] Integrate with LessonPage

### Phase 6: User Progress (1-2 hours)
- [ ] Track which story segments user has unlocked
- [ ] Update unlock status when lesson is completed
- [ ] Display locked segments as "coming soon"
- [ ] Show progress through story

### Milestones
| Milestone | Date | Status |
|-----------|------|--------|
| Database & Models | - | ⏳ |
| Backend Services | - | ⏳ |
| AI Integration | - | ⏳ |
| Audio Generation | - | ⏳ |
| Frontend Integration | - | ⏳ |
| User Progress | - | ⏳ |

---

## ✅ Tasks

### Backend
- [ ] Create Domain/Entities/StorySegment.cs
- [ ] Create Application/DTOs/StorySegmentDto.cs
- [ ] Create Application/DTOs/StoryGenerationRequest.cs
- [ ] Create Domain/Interfaces/IStoryRepository.cs
- [ ] Create Infrastructure/Data/Repositories/StoryRepository.cs
- [ ] Create Application/Services/StoryService.cs
- [ ] Create Application/Services/StoryGenerationService.cs
- [ ] Create Application/Services/MistralClientService.cs
- [ ] Create Presentation/Controllers/StoryController.cs
- [ ] Update Level and Lesson entities with StorySegment relationships
- [ ] Register services in Program.cs
- [ ] Write unit tests
- [ ] Write integration tests

### Database
- [ ] Create migration for StorySegments table
- [ ] Add foreign keys to Levels and Lessons
- [ ] Add indexes for LevelId, LessonId, Order
- [ ] Apply migration

### AI Integration
- [ ] Set up Mistral API client with configuration
- [ ] Create prompt templates for each CEFR level
- [ ] Implement vocabulary extraction from lessons
- [ ] Implement story segmentation logic
- [ ] Add error handling for Mistral API
- [ ] Add rate limiting/caching for Mistral calls

### Audio Generation
- [ ] Extend Coqui TTS service for story segments
- [ ] Implement batch audio generation
- [ ] Create audio file storage structure
- [ ] Add audio serving endpoint
- [ ] Implement audio cleanup mechanism

### Frontend
- [ ] Create pages/StoryPage.tsx
- [ ] Create components/StoryTab.tsx
- [ ] Create components/StorySegment.tsx
- [ ] Create components/StoryPlayer.tsx
- [ ] Create hooks/useStory.ts
- [ ] Create hooks/useStoryAudio.ts
- [ ] Implement word translation on click
- [ ] Add story progress indicator
- [ ] Add navigation between segments

### User Progress
- [ ] Extend UserProgress or create StoryProgress table
- [ ] Implement unlock logic when lesson completed
- [ ] Create endpoint to get user's story progress
- [ ] Add locked segment UI state

---

## 🔗 Dependencies

### Feature Dependencies
- [Infrastructure Setup](infrastructure-setup.md) - Required
- [Lesson Management](lesson-management.md) - Required (story segments associated with lessons/levels)
- [AI Services](ai-services.md) - Required (Mistral for generation, Coqui for audio)

### Technical Dependencies
- Mistral-Medium API access
- Coqui TTS Python library
- Entity Framework Core

### Blockers
- [ ] Infrastructure Setup must be complete
- [ ] Lesson Management must be complete
- [ ] AI Services must be configured
- [ ] Mistral API key required

---

## ✅ Definition of Done

### General Criteria (All Features)
- [ ] All acceptance criteria met and verified
- [ ] All tasks in this document completed
- [ ] Code follows Clean Architecture principles
- [ ] Code reviewed and approved by at least 1 team member
- [ ] All tests passing (unit, integration)
- [ ] Documentation updated (README, AGENTS.md if applicable)
- [ ] Feature works in development environment
- [ ] Feature deployed to staging environment
- [ ] Performance meets defined targets
- [ ] Security review completed
- [ ] No critical bugs or blockers

### Story-Specific Criteria
- [ ] Story segments can be created, read, updated, and deleted
- [ ] Stories are associated with levels and lessons
- [ ] Story content is generated using Mistral-Medium
- [ ] Story audio is generated using Coqui TTS
- [ ] Story segments unlock sequentially with lesson completion
- [ ] Users can view unlocked story segments
- [ ] Users can listen to story audio
- [ ] Users can click on words to see translations
- [ ] Story progress is tracked per user

---

## 🧪 Testing Strategy

### Testing Approach

| Test Type | Coverage | Tools | Responsibility |
|-----------|----------|-------|----------------|
| Unit Tests | 80%+ code coverage | MsTest, Moq | Backend Dev |
| Integration Tests | All service interactions | MsTest, TestContainers | Backend Dev |
| API Tests | All endpoints | MsTest, HttpClient | Backend Dev |
| Frontend Unit Tests | Component logic | Vitest | Frontend Dev |
| Frontend Integration | Service integration | Vitest | Frontend Dev |
| E2E Tests | Critical user journeys | Playwright | QA/Dev |
| Manual Testing | Exploratory, edge cases | BrowserStack | QA |

### Story-Specific Tests

#### Backend Tests
- [ ] Create story segment → success
- [ ] Create story segment with invalid level → error
- [ ] Get story segments by level → returns correct list
- [ ] Get story segment by ID → returns correct segment
- [ ] Update story segment → success
- [ ] Delete story segment → success
- [ ] Generate story with Mistral → returns valid story text
- [ ] Generate story with vocabulary list → includes all words
- [ ] Generate audio for story segment → creates audio file
- [ ] Get user's story progress → returns correct status

#### AI Tests
- [ ] Mistral generates valid German story for A1 level
- [ ] Mistral generates story with all requested vocabulary
- [ ] Mistral handles different CEFR levels appropriately
- [ ] Coqui TTS generates clear audio for story text
- [ ] Coqui TTS handles long story segments
- [ ] Coqui TTS handles German special characters (umlauts, ß)

#### Frontend Tests
- [ ] Story page loads and displays segments
- [ ] Audio plays for story segment
- [ ] Word click shows translation
- [ ] Navigation between segments works
- [ ] Locked segments are displayed as "coming soon"
- [ ] Story progress is displayed correctly

---

## 📝 Notes & Decisions

### Decisions Made
| Date | Decision | Rationale |
|------|----------|-----------|
| May 31, 2025 | One story per level for MVP | Simpler implementation, can expand later |
| May 31, 2025 | Segment per lesson | Natural progression, matches lesson structure |
| May 31, 2025 | Generate audio for all segments | Essential for accessibility and pronunciation |
| May 31, 2025 | Use Mistral-Medium | High-quality, cost-effective for this use case |
| May 31, 2025 | Click-to-translate words | Enhances learning without disrupting flow |

### Technical Notes
- StorySegments.LessonId can be NULL for intro/conclusion segments
- Order field ensures segments appear in correct sequence
- UNIQUE(LevelId, Order) prevents duplicate ordering
- Content should be stored as plain text, formatted on frontend
- Audio files should be named: `/audio/story/{levelId}-{order}.wav`

### Prompt Engineering
Different prompts for different levels:
- **A1**: Simple present tense, basic vocabulary, short sentences
- **A2**: Present, past, future tense, expanded vocabulary
- **B1**: Complex sentences, subordinate clauses, broader topics
- **B2/C1**: Advanced grammar, idiomatic expressions, nuanced topics

### Gotchas
- ⚠️ Mistral API has rate limits - implement caching
- ⚠️ AI generation may produce non-A1 vocabulary - need validation
- ⚠️ Long stories may exceed token limits - need segmentation
- ⚠️ Audio generation for long segments may be slow
- ⚠️ Need to handle cases where vocabulary list is empty

### Example Prompts by Level

**A1 Prompt:**
```
Write a 2-3 paragraph story about daily life in Germany for an A1 learner.
Use only A1 vocabulary and simple present tense sentences.
Include these words: {vocabularyList}
Theme: {theme}
```

**B1 Prompt:**
```
Write a 3-4 paragraph story about {theme} for a B1 German learner.
Use B1-level vocabulary and grammar including present, past, and future tenses.
Include these words: {vocabularyList}
Make the story engaging and suitable for adult learners.
```

---

## 📊 Progress History

| Date | Status Change | Notes |
|------|---------------|-------|
| May 31, 2025 | Created | Initial plan based on application-plan.md |

---

## 📎 Related Files & Links

- Architecture: [Backend Structure](../architecture/backend-structure.md)
- Architecture: [Application Plan](../architecture/application-plan.md)
- Database Schema: [Initial Database Schema](../database/initial-database-schema.sql)
- Feature: [Lesson Management](lesson-management.md)
- Feature: [AI Services](ai-services.md)
- Reference: [Mistral AI API](https://docs.mistral.ai/)
- Reference: [Coqui TTS](https://github.com/coqui-ai/TTS)

---

*Feature created from application-plan.md*