- Add comprehensive documentation in docs/ (architecture, features, roadmap) - Add german-app-frontend with Vite, TypeScript, ESLint configuration - Add AGENTS.md and .gitignore Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
1541 lines
No EOL
52 KiB
Markdown
1541 lines
No EOL
52 KiB
Markdown
# German Learning Web App - Comprehensive Plan
|
||
|
||
---
|
||
|
||
## **Table of Contents**
|
||
|
||
1. [Overview](#overview)
|
||
2. [Core Learning Experience](#core-learning-experience)
|
||
3. [Technical Stack](#technical-stack)
|
||
4. [Database Schema](#database-schema)
|
||
5. [Backend Architecture](#backend-architecture)
|
||
6. [Frontend Architecture](#frontend-architecture)
|
||
7. [AI Integration](#ai-integration)
|
||
8. [Speech Recognition and TTS](#speech-recognition-and-tts)
|
||
9. [Vocabulary Import](#vocabulary-import)
|
||
10. [TDD Approach](#tdd-approach)
|
||
11. [Development Roadmap](#development-roadmap)
|
||
12. [Example Workflows](#example-workflows)
|
||
13. [Deployment Notes](#deployment-notes)
|
||
14. [Open Questions](#open-questions)
|
||
|
||
---
|
||
|
||
## **1. Overview**
|
||
|
||
### **Purpose**
|
||
|
||
A web application designed to teach German from **A1 to C1 levels** with a structured, immersive, and interactive approach. The app focuses on **vocabulary, grammar, reading, writing, speaking, and listening** through a **linear learning path** with lessons, quizzes, and a continuous story.
|
||
|
||
### **Key Features**
|
||
|
||
- **Fixed Learning Path**: Linear progression from A1 to C1 (only A1 implemented initially).
|
||
- **Lessons**: Each lesson includes vocabulary, grammar, story segments, and interactive exercises.
|
||
- **Quizzes**: Static difficulty, unlimited retakes, 80% passing score.
|
||
- **Story Integration**: AI-generated story (Mistral-Medium) tied to lessons, unlocking segment by segment.
|
||
- **Speech Recognition**: Self-hosted [Vosk](https://alphacephei.com/vosk/) for speaking exercises.
|
||
- **Text-to-Speech (TTS)**: Self-hosted [Coqui TTS](https://github.com/coqui-ai/TTS) for audio generation.
|
||
- **Gamification**: Points, badges, and streaks to motivate users.
|
||
- **Progress Tracking**: Dashboard with visual progress, quiz scores, and weak area analysis.
|
||
- **Vocabulary Import**: Scraped from [Goethe Institut](https://www.goethe.de/en/spr/ueb.html) and [DW Learn German](https://learngerman.dw.com/en/learn-german/s-9528).
|
||
|
||
### **Target Audience**
|
||
|
||
- Self-learners of German at all levels (A1–C1).
|
||
- Users who prefer structured, immersive, and interactive learning.
|
||
|
||
---
|
||
|
||
## **2. Core Learning Experience**
|
||
|
||
### **A. Learning Path**
|
||
|
||
- **Levels**: A1 → A2 → B1 → B2 → C1 (only A1 implemented initially).
|
||
- **Progression**: Lessons **must be completed in order** within each level.
|
||
- **Unlocking**: Each lesson unlocks the next **only after passing its quiz** (80% score).
|
||
- **No Adaptive Difficulty**: Quizzes are static for all users.
|
||
|
||
### **B. Lesson Structure**
|
||
|
||
Each lesson follows this **immersive, audio-rich flow**:
|
||
|
||
|
||
| Step | Description | Audio/TTS Integration |
|
||
| ----------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------- |
|
||
| **1. Vocabulary** | 5–10 words (imported from Goethe/DW) with translations, articles (der/die/das), and images. | Coqui TTS generates audio for each word. |
|
||
| **2. Grammar** | Focus on one concept (e.g., nominative articles, present tense verbs). | Audio examples (e.g., "Der Mann **isst**"). |
|
||
| **3. Story** | AI-generated (Mistral-Medium) **daily life segment** using lesson vocabulary/grammar. | Coqui TTS generates audio for the story. |
|
||
| **4. Reading** | Short dialogue/text with comprehension questions. | Optional audio playback. |
|
||
| **5. Listening** | Play audio (e.g., a word, sentence, or dialogue) → user answers questions. | Vosk for user responses; Coqui TTS for prompts. |
|
||
| **6. Speaking** | User records themselves repeating phrases or answering prompts. | Vosk transcribes and validates responses. |
|
||
| **7. Writing** | Open-ended prompts (e.g., "Describe your morning using 3 new words."). | Mistral-Medium provides feedback. |
|
||
| **8. Quiz** | 10–15 questions: MCQ, fill-in-the-blank, **listening comprehension**, matching. | Coqui TTS for audio questions; Vosk for user. |
|
||
|
||
|
||
### **C. Story Integration**
|
||
|
||
- **Linear Story**: One continuous narrative per level (e.g., A1: *"A Week in the Life of Anna"*).
|
||
- **Generated by Mistral-Medium** with prompts like:
|
||
```
|
||
Write a 2–3 paragraph story about daily life for a German A1 learner.
|
||
Use only A1 vocabulary and grammar (present tense, basic nouns/verbs).
|
||
Include the words: [aufstehen, frühstücken, Arbeit].
|
||
Theme: "A typical morning."
|
||
```
|
||
- **Unlocked segment by segment** after completing lessons.
|
||
- **Audio**: Coqui TTS generates audio for each segment.
|
||
- **Interactive**: Click on words to see translations/definitions.
|
||
|
||
### **D. Quizzes**
|
||
|
||
- **Static Difficulty**: Same questions for all users.
|
||
- **Types**:
|
||
1. **Multiple Choice**: "What is the article for 'Buch'? a) der b) die c) das"
|
||
2. **Fill-in-the-Blank**: "Anna _____ um 7 Uhr auf." (steht)
|
||
3. **Listening Comprehension**:
|
||
- Play audio (e.g., *"Das ist ein Buch."*) → user selects the correct translation or image.
|
||
- Audio generated via Coqui TTS.
|
||
4. **Matching**: Drag words to their articles (der/die/das).
|
||
- **Passing**: 80% score to advance.
|
||
- **Retakes**: Unlimited, no penalties.
|
||
|
||
### **E. Progress Tracking**
|
||
|
||
- **User Dashboard**:
|
||
- Visual progress bar for current level (e.g., "A1: 3/10 lessons completed").
|
||
- Story progress (e.g., "Chapter 2/5 unlocked").
|
||
- Quiz scores history.
|
||
- Weak areas (e.g., "You often confuse 'die' and 'das'—try this exercise!").
|
||
- **Database**: `UserProgress` table tracks completed lessons, quiz scores, and timestamps.
|
||
|
||
---
|
||
|
||
## **3. Technical Stack**
|
||
|
||
|
||
| Component | Technology | Notes |
|
||
| ---------------------- | -------------------------------------------- | ----------------------------------------- |
|
||
| **Backend** | .NET 9.0 (ASP.NET Core) | |
|
||
| **Frontend** | React (TypeScript) + Vite | Mobile-first with Tailwind CSS |
|
||
| **Database** | PostgreSQL | |
|
||
| **Auth** | ASP.NET Core Identity + JWT | |
|
||
| **AI Integration** | Mistral-Medium API | Story/lesson generation, feedback |
|
||
| **Speech Recognition** | [Vosk](https://alphacephei.com/vosk/) | Model: `vosk-model-de-0.22` (mid-range) |
|
||
| **TTS** | [Coqui TTS](https://github.com/coqui-ai/TTS) | Open-source, self-hosted, supports German |
|
||
|
||
|
||
---
|
||
|
||
## **4. Database Schema**
|
||
|
||
### **SQL Script for PostgreSQL**
|
||
|
||
```sql
|
||
-- Enable UUID extension for unique identifiers
|
||
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
|
||
|
||
-- Users and Authentication
|
||
CREATE TABLE Users (
|
||
Id SERIAL PRIMARY KEY,
|
||
Username VARCHAR(50) UNIQUE NOT NULL,
|
||
Email VARCHAR(100) UNIQUE NOT NULL,
|
||
PasswordHash VARCHAR(255) NOT NULL,
|
||
CurrentLevel VARCHAR(10) DEFAULT 'A1',
|
||
Streak INT DEFAULT 0,
|
||
TotalPoints INT DEFAULT 0,
|
||
CreatedAt TIMESTAMP DEFAULT NOW()
|
||
);
|
||
|
||
-- User Progress Tracking
|
||
CREATE TABLE UserProgress (
|
||
Id SERIAL PRIMARY KEY,
|
||
UserId INT REFERENCES Users(Id) ON DELETE CASCADE,
|
||
LessonId INT,
|
||
IsCompleted BOOLEAN DEFAULT FALSE,
|
||
QuizScore INT,
|
||
LastAttemptDate TIMESTAMP DEFAULT NOW(),
|
||
UNIQUE(UserId, LessonId)
|
||
);
|
||
|
||
-- Learning Content: Levels
|
||
CREATE TABLE Levels (
|
||
Id SERIAL PRIMARY KEY,
|
||
Name VARCHAR(10) UNIQUE NOT NULL, -- e.g., "A1", "A2"
|
||
Description TEXT,
|
||
Order INT UNIQUE NOT NULL
|
||
);
|
||
|
||
-- Learning Content: Lessons
|
||
CREATE TABLE Lessons (
|
||
Id SERIAL PRIMARY KEY,
|
||
LevelId INT REFERENCES Levels(Id) ON DELETE CASCADE,
|
||
Title VARCHAR(100) NOT NULL,
|
||
Order INT NOT NULL,
|
||
Topic VARCHAR(100) NOT NULL,
|
||
UNIQUE(LevelId, Order)
|
||
);
|
||
|
||
-- Vocabulary
|
||
CREATE TABLE Vocabulary (
|
||
Id SERIAL PRIMARY KEY,
|
||
LessonId INT REFERENCES Lessons(Id) ON DELETE CASCADE,
|
||
Word VARCHAR(50) NOT NULL,
|
||
Translation VARCHAR(100) NOT NULL,
|
||
Article VARCHAR(10) CHECK (Article IN ('der', 'die', 'das', '')),
|
||
AudioUrl VARCHAR(255),
|
||
ImageUrl VARCHAR(255),
|
||
Source VARCHAR(50) CHECK (Source IN ('Goethe', 'DW'))
|
||
);
|
||
|
||
-- Story Segments
|
||
CREATE TABLE StorySegments (
|
||
Id SERIAL PRIMARY KEY,
|
||
LevelId INT REFERENCES Levels(Id) ON DELETE CASCADE,
|
||
LessonId INT REFERENCES Lessons(Id) ON DELETE SET NULL,
|
||
Content TEXT NOT NULL,
|
||
AudioUrl VARCHAR(255),
|
||
Order INT NOT NULL,
|
||
UNIQUE(LevelId, Order)
|
||
);
|
||
|
||
-- Quizzes
|
||
CREATE TABLE Quizzes (
|
||
Id SERIAL PRIMARY KEY,
|
||
LessonId INT REFERENCES Lessons(Id) ON DELETE CASCADE,
|
||
PassingScore INT DEFAULT 80
|
||
);
|
||
|
||
-- Quiz Questions
|
||
CREATE TABLE Questions (
|
||
Id SERIAL PRIMARY KEY,
|
||
QuizId INT REFERENCES Quizzes(Id) ON DELETE CASCADE,
|
||
Type VARCHAR(20) NOT NULL CHECK (Type IN ('mcq', 'fill_in', 'listening', 'matching')),
|
||
Content TEXT NOT NULL,
|
||
CorrectAnswer TEXT NOT NULL,
|
||
AudioUrl VARCHAR(255),
|
||
Options TEXT[], -- For MCQ: ['option1', 'option2']
|
||
Points INT DEFAULT 1
|
||
);
|
||
|
||
-- Gamification: Badges
|
||
CREATE TABLE Badges (
|
||
Id SERIAL PRIMARY KEY,
|
||
Name VARCHAR(50) NOT NULL,
|
||
Description TEXT,
|
||
ImageUrl VARCHAR(255),
|
||
UNIQUE(Name)
|
||
);
|
||
|
||
-- User Badges
|
||
CREATE TABLE UserBadges (
|
||
UserId INT REFERENCES Users(Id) ON DELETE CASCADE,
|
||
BadgeId INT REFERENCES Badges(Id) ON DELETE CASCADE,
|
||
EarnedDate TIMESTAMP DEFAULT NOW(),
|
||
PRIMARY KEY (UserId, BadgeId)
|
||
);
|
||
|
||
-- Practice Sessions (Speaking/Writing)
|
||
CREATE TABLE PracticeSessions (
|
||
Id SERIAL PRIMARY KEY,
|
||
UserId INT REFERENCES Users(Id) ON DELETE CASCADE,
|
||
Type VARCHAR(20) NOT NULL CHECK (Type IN ('speaking', 'writing')),
|
||
Prompt TEXT NOT NULL,
|
||
UserResponse TEXT,
|
||
Feedback TEXT,
|
||
Date TIMESTAMP DEFAULT NOW()
|
||
);
|
||
|
||
-- Audio Files Metadata (Optional)
|
||
CREATE TABLE AudioFiles (
|
||
Id SERIAL PRIMARY KEY,
|
||
Text TEXT NOT NULL,
|
||
FilePath VARCHAR(255) NOT NULL,
|
||
Type VARCHAR(20) NOT NULL CHECK (Type IN ('vocabulary', 'story', 'quiz')),
|
||
CreatedAt TIMESTAMP DEFAULT NOW(),
|
||
UNIQUE(Text, Type)
|
||
);
|
||
|
||
-- Indexes for Performance
|
||
CREATE INDEX idx_userprogress_userid ON UserProgress(UserId);
|
||
CREATE INDEX idx_vocabulary_lessonid ON Vocabulary(LessonId);
|
||
CREATE INDEX idx_questions_quizid ON Questions(QuizId);
|
||
```
|
||
|
||
---
|
||
|
||
## **5. Backend Architecture**
|
||
|
||
### **Project Structure**
|
||
|
||
```
|
||
GermanApp/
|
||
├── Controllers/
|
||
│ ├── AuthController.cs
|
||
│ ├── LessonsController.cs
|
||
│ ├── QuizzesController.cs
|
||
│ ├── StoryController.cs
|
||
│ ├── SpeechController.cs
|
||
│ ├── TtsController.cs
|
||
│ └── VocabularyController.cs
|
||
├── Services/
|
||
│ ├── AuthService.cs
|
||
│ ├── LessonService.cs
|
||
│ ├── QuizService.cs
|
||
│ ├── StoryService.cs
|
||
│ ├── VoskService.cs
|
||
│ ├── TtsService.cs
|
||
│ ├── VocabularyImportService.cs
|
||
│ └── interfaces/
|
||
│ ├── IVoskService.cs
|
||
│ ├── ITtsService.cs
|
||
│ └── ...
|
||
├── Models/
|
||
│ ├── User.cs
|
||
│ ├── Lesson.cs
|
||
│ ├── Vocabulary.cs
|
||
│ ├── Quiz.cs
|
||
│ ├── StorySegment.cs
|
||
│ ├── DTOs/
|
||
│ │ ├── LessonDto.cs
|
||
│ │ ├── QuizDto.cs
|
||
│ │ └── ...
|
||
│ └── AppDbContext.cs
|
||
├── Data/
|
||
│ └── AppDbContext.cs
|
||
├── Tests/
|
||
│ ├── Unit/
|
||
│ │ ├── Services/
|
||
│ │ │ ├── VoskServiceTests.cs
|
||
│ │ │ ├── TtsServiceTests.cs
|
||
│ │ │ └── ...
|
||
│ │ └── Controllers/
|
||
│ │ ├── LessonsControllerTests.cs
|
||
│ │ └── ...
|
||
│ └── Integration/
|
||
│ ├── LessonsControllerIntegrationTests.cs
|
||
│ └── ...
|
||
├── appsettings.json
|
||
├── Program.cs
|
||
└── GermanApp.csproj
|
||
```
|
||
|
||
### **Key Services**
|
||
|
||
#### **1. `IVoskService` and `VoskService**`
|
||
|
||
```csharp
|
||
public interface IVoskService
|
||
{
|
||
Task<string> RecognizeSpeechAsync(byte[] audioData);
|
||
}
|
||
|
||
public class VoskService : IVoskService
|
||
{
|
||
private readonly string _modelPath;
|
||
|
||
public VoskService(IConfiguration configuration)
|
||
{
|
||
_modelPath = configuration["Vosk:ModelPath"];
|
||
}
|
||
|
||
public async Task<string> RecognizeSpeechAsync(byte[] audioData)
|
||
{
|
||
// Use Vosk .NET wrapper or CLI
|
||
// Example using CLI (simplified)
|
||
var tempFile = Path.GetTempFileName();
|
||
await File.WriteAllBytesAsync(tempFile, audioData);
|
||
|
||
var process = new System.Diagnostics.Process
|
||
{
|
||
StartInfo = new System.Diagnostics.ProcessStartInfo
|
||
{
|
||
FileName = "python",
|
||
Arguments = $"-m vosk.transcribe --model {_modelPath} --input {tempFile}",
|
||
RedirectStandardOutput = true,
|
||
UseShellExecute = false,
|
||
CreateNoWindow = true
|
||
}
|
||
};
|
||
process.Start();
|
||
var result = await process.StandardOutput.ReadToEndAsync();
|
||
await process.WaitForExitAsync();
|
||
|
||
return result.Trim();
|
||
}
|
||
}
|
||
```
|
||
|
||
#### **2. `ITtsService` and `TtsService**`
|
||
|
||
```csharp
|
||
public interface ITtsService
|
||
{
|
||
Task<byte[]> GenerateTtsAsync(string text);
|
||
}
|
||
|
||
public class TtsService : ITtsService
|
||
{
|
||
private readonly string _pythonPath;
|
||
private readonly string _coquiModelName;
|
||
|
||
public TtsService(IConfiguration configuration)
|
||
{
|
||
_pythonPath = configuration["Coqui:PythonPath"];
|
||
_coquiModelName = configuration["Coqui:ModelName"];
|
||
}
|
||
|
||
public async Task<byte[]> GenerateTtsAsync(string text)
|
||
{
|
||
var tempOutputFile = Path.GetTempFileName() + ".wav";
|
||
|
||
var process = new System.Diagnostics.Process
|
||
{
|
||
StartInfo = new System.Diagnostics.ProcessStartInfo
|
||
{
|
||
FileName = _pythonPath,
|
||
Arguments = $"-m TTS.api --model {_coquiModelName} --text \"{text}\" --out_path {tempOutputFile}",
|
||
RedirectStandardOutput = true,
|
||
UseShellExecute = false,
|
||
CreateNoWindow = true
|
||
}
|
||
};
|
||
process.Start();
|
||
await process.WaitForExitAsync();
|
||
|
||
return await File.ReadAllBytesAsync(tempOutputFile);
|
||
}
|
||
}
|
||
```
|
||
|
||
#### **3. `IStoryService` and `StoryService**`
|
||
|
||
```csharp
|
||
public interface IStoryService
|
||
{
|
||
Task<string> GenerateStorySegmentAsync(List<string> vocabulary, string theme, string level);
|
||
}
|
||
|
||
public class StoryService : IStoryService
|
||
{
|
||
private readonly HttpClient _httpClient;
|
||
private readonly string _mistralApiKey;
|
||
|
||
public StoryService(HttpClient httpClient, IConfiguration configuration)
|
||
{
|
||
_httpClient = httpClient;
|
||
_mistralApiKey = configuration["Mistral:ApiKey"];
|
||
}
|
||
|
||
public async Task<string> GenerateStorySegmentAsync(List<string> vocabulary, string theme, string level)
|
||
{
|
||
var prompt = $@"
|
||
Write a 2–3 paragraph story about daily life for a German {level} learner.
|
||
Use only {level} vocabulary and grammar (present tense, basic nouns/verbs).
|
||
Include the following words: {string.Join(", ", vocabulary)}.
|
||
Theme: "{theme}".
|
||
";
|
||
|
||
var request = new
|
||
{
|
||
model = "mistral-medium",
|
||
messages = new[] { new { role = "user", content = prompt } }
|
||
};
|
||
|
||
_httpClient.DefaultRequestHeaders.Authorization =
|
||
new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", _mistralApiKey);
|
||
|
||
var response = await _httpClient.PostAsJsonAsync("https://api.mistral.ai/v1/chat/completions", request);
|
||
var result = await response.Content.ReadFromJsonAsync<MistralResponse>();
|
||
|
||
return result.Choices[0].Message.Content;
|
||
}
|
||
}
|
||
|
||
public class MistralResponse
|
||
{
|
||
public Choice[] Choices { get; set; }
|
||
}
|
||
|
||
public class Choice
|
||
{
|
||
public Message Message { get; set; }
|
||
}
|
||
|
||
public class Message
|
||
{
|
||
public string Content { get; set; }
|
||
}
|
||
```
|
||
|
||
#### **4. `IQuizService` and `QuizService**`
|
||
|
||
```csharp
|
||
public interface IQuizService
|
||
{
|
||
Task<QuizResult> SubmitQuizAsync(int userId, int quizId, List<AnswerDto> answers);
|
||
}
|
||
|
||
public class QuizService : IQuizService
|
||
{
|
||
private readonly AppDbContext _dbContext;
|
||
|
||
public QuizService(AppDbContext dbContext)
|
||
{
|
||
_dbContext = dbContext;
|
||
}
|
||
|
||
public async Task<QuizResult> SubmitQuizAsync(int userId, int quizId, List<AnswerDto> answers)
|
||
{
|
||
var quiz = await _dbContext.Quizzes
|
||
.Include(q => q.Questions)
|
||
.FirstOrDefaultAsync(q => q.Id == quizId);
|
||
|
||
if (quiz == null)
|
||
throw new NotFoundException("Quiz not found.");
|
||
|
||
var score = 0;
|
||
var totalPoints = quiz.Questions.Sum(q => q.Points);
|
||
|
||
foreach (var answer in answers)
|
||
{
|
||
var question = quiz.Questions.FirstOrDefault(q => q.Id == answer.QuestionId);
|
||
if (question != null && question.CorrectAnswer == answer.UserAnswer)
|
||
{
|
||
score += question.Points;
|
||
}
|
||
}
|
||
|
||
var passed = (score * 100 / totalPoints) >= quiz.PassingScore;
|
||
|
||
// Update user progress
|
||
var userProgress = await _dbContext.UserProgress
|
||
.FirstOrDefaultAsync(up => up.UserId == userId && up.LessonId == quiz.LessonId);
|
||
|
||
if (userProgress == null)
|
||
{
|
||
userProgress = new UserProgress
|
||
{
|
||
UserId = userId,
|
||
LessonId = quiz.LessonId,
|
||
IsCompleted = passed,
|
||
QuizScore = score,
|
||
LastAttemptDate = DateTime.UtcNow
|
||
};
|
||
_dbContext.UserProgress.Add(userProgress);
|
||
}
|
||
else
|
||
{
|
||
userProgress.IsCompleted = passed;
|
||
userProgress.QuizScore = score;
|
||
userProgress.LastAttemptDate = DateTime.UtcNow;
|
||
}
|
||
|
||
await _dbContext.SaveChangesAsync();
|
||
|
||
return new QuizResult
|
||
{
|
||
Passed = passed,
|
||
Score = score,
|
||
TotalPoints = totalPoints
|
||
};
|
||
}
|
||
}
|
||
|
||
public class QuizResult
|
||
{
|
||
public bool Passed { get; set; }
|
||
public int Score { get; set; }
|
||
public int TotalPoints { get; set; }
|
||
}
|
||
|
||
public class AnswerDto
|
||
{
|
||
public int QuestionId { get; set; }
|
||
public string UserAnswer { get; set; }
|
||
}
|
||
```
|
||
|
||
### **API Endpoints**
|
||
|
||
|
||
| Endpoint | Method | Description | Request Body | Response |
|
||
| ------------------------------- | ------ | ------------------------------- | ------------------------------------------- | -------------------------------------------- |
|
||
| `/api/auth/register` | POST | Register a new user. | `{ username, email, password }` | `{ userId, token }` |
|
||
| `/api/auth/login` | POST | Log in a user. | `{ email, password }` | `{ userId, token }` |
|
||
| `/api/auth/me` | GET | Get current user details. | - | `{ user }` |
|
||
| `/api/levels` | GET | List all levels. | - | `[{ level }]` |
|
||
| `/api/levels/{levelId}/lessons` | GET | List lessons for a level. | - | `[{ lesson }]` |
|
||
| `/api/lessons/{id}` | GET | Get lesson details. | - | `{ lesson, vocabulary, storySegment, quiz }` |
|
||
| `/api/quizzes/{quizId}` | GET | Get quiz questions. | - | `{ quiz, questions }` |
|
||
| `/api/quizzes/{quizId}/submit` | POST | Submit quiz answers. | `{ answers: [{ questionId, userAnswer }] }` | `{ passed, score, totalPoints }` |
|
||
| `/api/story/{levelId}/segments` | GET | Get story segments for a level. | - | `[{ storySegment }]` |
|
||
| `/api/speech/recognize` | POST | Recognize speech. | `{ audio: File }` | `{ text }` |
|
||
| `/api/tts/generate` | POST | Generate TTS audio. | `{ text }` | `{ audio: File }` |
|
||
| `/api/ai/generate-story` | POST | Generate a story segment. | `{ vocabulary, theme, level }` | `{ storyText }` |
|
||
| `/api/ai/feedback` | POST | Get feedback for writing. | `{ userInput, level }` | `{ corrected, explanation, encouragement }` |
|
||
| `/api/vocabulary/import` | POST | Import vocabulary (admin). | - | `{ success }` |
|
||
|
||
|
||
---
|
||
|
||
## **6. Frontend Architecture**
|
||
|
||
### **Project Structure**
|
||
|
||
```
|
||
german-app-frontend/
|
||
├── public/
|
||
│ └── audio/ -- Static audio files (vocabulary, story, quizzes)
|
||
├── src/
|
||
│ ├── components/
|
||
│ │ ├── Dashboard/
|
||
│ │ │ ├── ProgressBar.tsx
|
||
│ │ │ ├── NextLessonButton.tsx
|
||
│ │ │ └── StoryProgress.tsx
|
||
│ │ ├── Lesson/
|
||
│ │ │ ├── VocabularyTab.tsx
|
||
│ │ │ ├── GrammarTab.tsx
|
||
│ │ │ ├── StoryTab.tsx
|
||
│ │ │ ├── ListeningExercise.tsx
|
||
│ │ │ ├── SpeakingExercise.tsx
|
||
│ │ │ ├── WritingExercise.tsx
|
||
│ │ │ └── QuizTab.tsx
|
||
│ │ ├── Quiz/
|
||
│ │ │ ├── McqQuestion.tsx
|
||
│ │ │ ├── FillInBlank.tsx
|
||
│ │ │ ├── ListeningQuestion.tsx
|
||
│ │ │ └── QuizResults.tsx
|
||
│ │ ├── Story/
|
||
│ │ │ ├── StorySegment.tsx
|
||
│ │ │ └── StoryPlayer.tsx
|
||
│ │ ├── Shared/
|
||
│ │ │ ├── AudioPlayer.tsx
|
||
│ │ │ ├── Recorder.tsx
|
||
│ │ │ ├── Button.tsx
|
||
│ │ │ └── types.ts
|
||
│ │ └── Layout/
|
||
│ │ ├── Header.tsx
|
||
│ │ └── Footer.tsx
|
||
│ ├── pages/
|
||
│ │ ├── DashboardPage.tsx
|
||
│ │ ├── LessonPage.tsx
|
||
│ │ ├── QuizPage.tsx
|
||
│ │ ├── StoryPage.tsx
|
||
│ │ ├── PracticePage.tsx
|
||
│ │ └── ProfilePage.tsx
|
||
│ ├── hooks/
|
||
│ │ ├── useAudio.ts
|
||
│ │ ├── useRecorder.ts
|
||
│ │ ├── useApi.ts
|
||
│ │ └── useAuth.ts
|
||
│ ├── services/
|
||
│ │ ├── api.ts
|
||
│ │ ├── authService.ts
|
||
│ │ ├── lessonService.ts
|
||
│ │ ├── quizService.ts
|
||
│ │ ├── speechService.ts
|
||
│ │ └── ttsService.ts
|
||
│ ├── styles/
|
||
│ │ ├── tailwind.css
|
||
│ │ └── global.css
|
||
│ ├── utils/
|
||
│ │ ├── constants.ts
|
||
│ │ └── helpers.ts
|
||
│ ├── types/
|
||
│ │ └── index.ts
|
||
│ ├── App.tsx
|
||
│ └── main.tsx
|
||
├── package.json
|
||
├── vite.config.ts
|
||
└── tsconfig.json
|
||
```
|
||
|
||
### **Key Components**
|
||
|
||
#### **1. `AudioPlayer.tsx**`
|
||
|
||
```tsx
|
||
import React, { useRef } from 'react';
|
||
|
||
interface AudioPlayerProps {
|
||
audioUrl: string;
|
||
className?: string;
|
||
}
|
||
|
||
export const AudioPlayer: React.FC<AudioPlayerProps> = ({ audioUrl, className }) => {
|
||
const audioRef = useRef<HTMLAudioElement>(null);
|
||
|
||
const handlePlay = () => {
|
||
audioRef.current?.play();
|
||
};
|
||
|
||
return (
|
||
<div className={className}>
|
||
<audio ref={audioRef} src={audioUrl} />
|
||
<button onClick={handlePlay} className="bg-blue-500 text-white p-2 rounded">
|
||
▶️ Play
|
||
</button>
|
||
</div>
|
||
);
|
||
};
|
||
```
|
||
|
||
#### **2. `Recorder.tsx**`
|
||
|
||
```tsx
|
||
import React, { useState, useRef } from 'react';
|
||
|
||
interface RecorderProps {
|
||
onTranscript: (text: string) => void;
|
||
className?: string;
|
||
}
|
||
|
||
export const Recorder: React.FC<RecorderProps> = ({ onTranscript, className }) => {
|
||
const [isRecording, setIsRecording] = useState(false);
|
||
const mediaRecorderRef = useRef<MediaRecorder | null>(null);
|
||
const audioChunksRef = useRef<Blob[]>([]);
|
||
|
||
const startRecording = async () => {
|
||
try {
|
||
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
||
mediaRecorderRef.current = new MediaRecorder(stream);
|
||
audioChunksRef.current = [];
|
||
|
||
mediaRecorderRef.current.ondataavailable = (event) => {
|
||
audioChunksRef.current.push(event.data);
|
||
};
|
||
|
||
mediaRecorderRef.current.onstop = async () => {
|
||
const audioBlob = new Blob(audioChunksRef.current, { type: 'audio/wav' });
|
||
const formData = new FormData();
|
||
formData.append('audio', audioBlob, 'recording.wav');
|
||
|
||
const response = await fetch('/api/speech/recognize', {
|
||
method: 'POST',
|
||
body: formData,
|
||
});
|
||
const { text } = await response.json();
|
||
onTranscript(text);
|
||
};
|
||
|
||
mediaRecorderRef.current.start();
|
||
setIsRecording(true);
|
||
} catch (error) {
|
||
console.error('Error accessing microphone:', error);
|
||
}
|
||
};
|
||
|
||
const stopRecording = () => {
|
||
if (mediaRecorderRef.current) {
|
||
mediaRecorderRef.current.stop();
|
||
setIsRecording(false);
|
||
}
|
||
};
|
||
|
||
return (
|
||
<div className={className}>
|
||
<button
|
||
onClick={isRecording ? stopRecording : startRecording}
|
||
className={`p-2 rounded ${isRecording ? 'bg-red-500' : 'bg-green-500'} text-white`}
|
||
>
|
||
{isRecording ? '⏹️ Stop Recording' : '🎤 Start Recording'}
|
||
</button>
|
||
</div>
|
||
);
|
||
};
|
||
```
|
||
|
||
#### **3. `ListeningQuestion.tsx**`
|
||
|
||
```tsx
|
||
import React, { useRef, useState } from 'react';
|
||
import { AudioPlayer } from '../Shared/AudioPlayer';
|
||
|
||
interface ListeningQuestionProps {
|
||
audioUrl: string;
|
||
options: string[];
|
||
correctAnswer: string;
|
||
onAnswer: (isCorrect: boolean) => void;
|
||
className?: string;
|
||
}
|
||
|
||
export const ListeningQuestion: React.FC<ListeningQuestionProps> = ({
|
||
audioUrl,
|
||
options,
|
||
correctAnswer,
|
||
onAnswer,
|
||
className
|
||
}) => {
|
||
const [selected, setSelected] = useState<string | null>(null);
|
||
|
||
const handleSelect = (option: string) => {
|
||
setSelected(option);
|
||
onAnswer(option === correctAnswer);
|
||
};
|
||
|
||
return (
|
||
<div className={className}>
|
||
<AudioPlayer audioUrl={audioUrl} />
|
||
<div className="mt-4">
|
||
{options.map((option) => (
|
||
<button
|
||
key={option}
|
||
onClick={() => handleSelect(option)}
|
||
disabled={selected !== null}
|
||
className={`block w-full mb-2 p-2 rounded ${
|
||
selected === option
|
||
? option === correctAnswer
|
||
? 'bg-green-500'
|
||
: 'bg-red-500'
|
||
: 'bg-blue-500'
|
||
} text-white`}
|
||
>
|
||
{option}
|
||
</button>
|
||
))}
|
||
</div>
|
||
</div>
|
||
);
|
||
};
|
||
```
|
||
|
||
#### **4. `LessonPage.tsx**`
|
||
|
||
```tsx
|
||
import React, { useState, useEffect } from 'react';
|
||
import { useParams } from 'react-router-dom';
|
||
import { VocabularyTab } from './VocabularyTab';
|
||
import { GrammarTab } from './GrammarTab';
|
||
import { StoryTab } from './StoryTab';
|
||
import { QuizTab } from '../Quiz/QuizTab';
|
||
|
||
interface Lesson {
|
||
id: number;
|
||
title: string;
|
||
vocabulary: Vocabulary[];
|
||
grammar: Grammar;
|
||
storySegment: StorySegment;
|
||
quizId: number;
|
||
}
|
||
|
||
interface Vocabulary {
|
||
id: number;
|
||
word: string;
|
||
translation: string;
|
||
article: string;
|
||
audioUrl: string;
|
||
}
|
||
|
||
interface Grammar {
|
||
topic: string;
|
||
explanation: string;
|
||
examples: string[];
|
||
}
|
||
|
||
interface StorySegment {
|
||
id: number;
|
||
content: string;
|
||
audioUrl: string;
|
||
}
|
||
|
||
export const LessonPage: React.FC = () => {
|
||
const { lessonId } = useParams<{ lessonId: string }>();
|
||
const [lesson, setLesson] = useState<Lesson | null>(null);
|
||
const [activeTab, setActiveTab] = useState<'vocabulary' | 'grammar' | 'story' | 'quiz'>('vocabulary');
|
||
|
||
useEffect(() => {
|
||
const fetchLesson = async () => {
|
||
const response = await fetch(`/api/lessons/${lessonId}`);
|
||
const data = await response.json();
|
||
setLesson(data);
|
||
};
|
||
fetchLesson();
|
||
}, [lessonId]);
|
||
|
||
if (!lesson) return <div>Loading...</div>;
|
||
|
||
return (
|
||
<div className="container mx-auto p-4">
|
||
<h1 className="text-2xl font-bold mb-4">{lesson.title}</h1>
|
||
|
||
<div className="flex mb-4 border-b">
|
||
<button
|
||
onClick={() => setActiveTab('vocabulary')}
|
||
className={`px-4 py-2 ${activeTab === 'vocabulary' ? 'border-b-2 border-blue-500' : ''}`}
|
||
>
|
||
Vocabulary
|
||
</button>
|
||
<button
|
||
onClick={() => setActiveTab('grammar')}
|
||
className={`px-4 py-2 ${activeTab === 'grammar' ? 'border-b-2 border-blue-500' : ''}`}
|
||
>
|
||
Grammar
|
||
</button>
|
||
<button
|
||
onClick={() => setActiveTab('story')}
|
||
className={`px-4 py-2 ${activeTab === 'story' ? 'border-b-2 border-blue-500' : ''}`}
|
||
>
|
||
Story
|
||
</button>
|
||
<button
|
||
onClick={() => setActiveTab('quiz')}
|
||
className={`px-4 py-2 ${activeTab === 'quiz' ? 'border-b-2 border-blue-500' : ''}`}
|
||
>
|
||
Quiz
|
||
</button>
|
||
</div>
|
||
|
||
<div className="p-4 border rounded-lg">
|
||
{activeTab === 'vocabulary' && (
|
||
<VocabularyTab vocabulary={lesson.vocabulary} />
|
||
)}
|
||
{activeTab === 'grammar' && (
|
||
<GrammarTab grammar={lesson.grammar} />
|
||
)}
|
||
{activeTab === 'story' && (
|
||
<StoryTab segment={lesson.storySegment} />
|
||
)}
|
||
{activeTab === 'quiz' && (
|
||
<QuizTab quizId={lesson.quizId} />
|
||
)}
|
||
</div>
|
||
</div>
|
||
);
|
||
};
|
||
```
|
||
|
||
---
|
||
|
||
## **7. AI Integration**
|
||
|
||
### **Mistral-Medium API**
|
||
|
||
- **Base URL**: `https://api.mistral.ai/v1/`
|
||
- **Authentication**: Bearer token (`MISTRAL_API_KEY`).
|
||
|
||
#### **Prompts**
|
||
|
||
##### **1. Story Generation**
|
||
|
||
```
|
||
Write a 2–3 paragraph story about daily life for a German {level} learner.
|
||
Use only {level} vocabulary and grammar (present tense, basic nouns/verbs like 'gehen', 'einkaufen', 'essen').
|
||
Include the following words: {vocabularyList}.
|
||
Theme: "{theme}".
|
||
|
||
Example for A1:
|
||
- Vocabulary: ["aufstehen", "frühstücken", "Arbeit", "schlafen"]
|
||
- Theme: "A typical morning"
|
||
```
|
||
|
||
**Output Example**:
|
||
|
||
```
|
||
Anna steht jeden Morgen um 7 Uhr auf. Sie geht ins Badezimmer und wäscht sich das Gesicht.
|
||
Dann frühstückt sie in der Küche. Sie isst ein Brot und trinkt einen Kaffee.
|
||
Um 8 Uhr geht Anna zur Arbeit. Sie arbeitet bis 17 Uhr. Am Abend kocht sie Abendessen und schläft um 22 Uhr.
|
||
```
|
||
|
||
##### **2. Writing Feedback**
|
||
|
||
```
|
||
The user wrote the following German sentence: "{userInput}".
|
||
They are an {level} learner. Provide:
|
||
1. A corrected version of the sentence.
|
||
2. A brief explanation of mistakes (in English).
|
||
3. Encouragement (e.g., "Good try! Remember that nouns are always capitalized.").
|
||
|
||
Example for A1:
|
||
- User Input: "ich stehe um 7 uhr auf."
|
||
```
|
||
|
||
**Output Example**:
|
||
|
||
```json
|
||
{
|
||
"corrected": "Ich stehe um 7 Uhr auf.",
|
||
"explanation": "'Ich' and 'Uhr' must be capitalized in German. Also, '7 Uhr' should be written as '7 Uhr' (no space).",
|
||
"encouragement": "Good try! Remember that all nouns and the word 'Ich' are always capitalized in German."
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## **8. Speech Recognition and TTS**
|
||
|
||
### **A. Vosk Speech Recognition**
|
||
|
||
- **Model**: `vosk-model-de-0.22` (mid-range, ~500MB).
|
||
- **Download**: [Vosk Models](https://alphacephei.com/vosk/models)
|
||
- **Integration**:
|
||
- **Backend**: Use the [Vosk .NET wrapper](https://github.com/alphacep/vosk-api) or call Vosk via Python CLI.
|
||
- **Frontend**: Record audio using the `Recorder` component and send it to `/api/speech/recognize`.
|
||
|
||
#### **Example Workflow**
|
||
|
||
1. User clicks "Record" in the `SpeakingExercise` component.
|
||
2. Frontend records audio and sends it to `/api/speech/recognize`.
|
||
3. Backend uses Vosk to transcribe the audio.
|
||
4. Backend returns the transcribed text (e.g., "Ich stehe um 8 Uhr auf.").
|
||
5. Frontend compares the transcription to the expected answer and provides feedback.
|
||
|
||
### **B. Coqui TTS**
|
||
|
||
- **Model**: `tts_models/de/deu/fairseq/vits` (high-quality German TTS).
|
||
- **Installation**:
|
||
```bash
|
||
pip install TTS
|
||
python -m TTS --download_model tts_models/de/deu/fairseq/vits
|
||
```
|
||
- **Integration**:
|
||
- **Backend**: Call Coqui TTS via Python CLI to generate audio for vocabulary, story segments, and quiz questions.
|
||
- **Frontend**: Play generated audio using the `AudioPlayer` component.
|
||
|
||
#### **Example Workflow**
|
||
|
||
1. Backend calls Coqui TTS to generate audio for a vocabulary word (e.g., "aufstehen").
|
||
2. Backend saves the audio file to `/public/audio/vocabulary/{id}.wav`.
|
||
3. Frontend plays the audio when the user clicks the "Play" button.
|
||
|
||
---
|
||
|
||
## **9. Vocabulary Import**
|
||
|
||
### **Sources**
|
||
|
||
1. [Goethe Institut A1 Word List](https://www.goethe.de/en/spr/ueb.html)
|
||
2. [DW Learn German A1 Vocabulary](https://learngerman.dw.com/en/learn-german/s-9528)
|
||
|
||
### **Implementation**
|
||
|
||
1. **Scraping Script**:
|
||
- Use a .NET HTTP client (e.g., `HttpClient`) to fetch and parse vocabulary from Goethe/DW.
|
||
- Store words in the `Vocabulary` table with their translations, articles, and source.
|
||
2. **Audio Generation**:
|
||
- For each word, call Coqui TTS to generate audio and save the `AudioUrl`.
|
||
3. **Admin Endpoint**:
|
||
- `POST /api/vocabulary/import` (admin-only).
|
||
|
||
#### **Example Scraping Logic (Pseudocode)**
|
||
|
||
```csharp
|
||
public class VocabularyImportService : IVocabularyImportService
|
||
{
|
||
private readonly ITtsService _ttsService;
|
||
private readonly AppDbContext _dbContext;
|
||
private readonly HttpClient _httpClient;
|
||
|
||
public async Task ImportVocabularyAsync()
|
||
{
|
||
var goetheWords = await ScrapeGoetheA1Words();
|
||
var dwWords = await ScrapeDwA1Words();
|
||
|
||
foreach (var word in goetheWords.Concat(dwWords))
|
||
{
|
||
var audioBytes = await _ttsService.GenerateTtsAsync(word.German);
|
||
var audioUrl = $"/audio/vocabulary/{Guid.NewGuid()}.wav";
|
||
await File.WriteAllBytesAsync(audioUrl, audioBytes);
|
||
|
||
_dbContext.Vocabulary.Add(new Vocabulary
|
||
{
|
||
Word = word.German,
|
||
Translation = word.English,
|
||
Article = word.Article,
|
||
AudioUrl = audioUrl,
|
||
Source = word.Source
|
||
});
|
||
}
|
||
await _dbContext.SaveChangesAsync();
|
||
}
|
||
|
||
private async Task<List<Word>> ScrapeGoetheA1Words() { /* ... */ }
|
||
private async Task<List<Word>> ScrapeDwA1Words() { /* ... */ }
|
||
}
|
||
|
||
public class Word
|
||
{
|
||
public string German { get; set; }
|
||
public string English { get; set; }
|
||
public string Article { get; set; }
|
||
public string Source { get; set; } // "Goethe" or "DW"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## **10. TDD Approach**
|
||
|
||
### **Backend Tests (.NET)**
|
||
|
||
- **Framework**: xUnit or NUnit.
|
||
- **Mocking**: Moq for mocking dependencies (e.g., Vosk, Coqui TTS, Mistral-Medium).
|
||
|
||
#### **Example Tests**
|
||
|
||
##### **1. VoskService Tests**
|
||
|
||
```csharp
|
||
using Moq;
|
||
using Xunit;
|
||
|
||
public class VoskServiceTests
|
||
{
|
||
[Fact]
|
||
public async Task RecognizeSpeech_ReturnsCorrectTranscription()
|
||
{
|
||
// Arrange
|
||
var mockVoskWrapper = new Mock<IVoskWrapper>();
|
||
mockVoskWrapper.Setup(w => w.Recognize(It.IsAny<byte[]>()))
|
||
.Returns("Guten Morgen");
|
||
var service = new VoskService(mockVoskWrapper.Object);
|
||
|
||
// Act
|
||
var result = await service.RecognizeSpeechAsync(new byte[100]);
|
||
|
||
// Assert
|
||
Assert.Equal("Guten Morgen", result);
|
||
}
|
||
|
||
[Fact]
|
||
public async Task RecognizeSpeech_HandlesEmptyAudio()
|
||
{
|
||
// Arrange
|
||
var mockVoskWrapper = new Mock<IVoskWrapper>();
|
||
mockVoskWrapper.Setup(w => w.Recognize(It.IsAny<byte[]>()))
|
||
.Throws<Exception>();
|
||
var service = new VoskService(mockVoskWrapper.Object);
|
||
|
||
// Act & Assert
|
||
await Assert.ThrowsAsync<Exception>(() => service.RecognizeSpeechAsync(new byte[0]));
|
||
}
|
||
}
|
||
```
|
||
|
||
##### **2. TtsService Tests**
|
||
|
||
```csharp
|
||
public class TtsServiceTests
|
||
{
|
||
[Fact]
|
||
public async Task GenerateTts_ReturnsAudioFile()
|
||
{
|
||
// Arrange
|
||
var mockCoquiWrapper = new Mock<ICoquiWrapper>();
|
||
mockCoquiWrapper.Setup(w => w.GenerateTts(It.IsAny<string>()))
|
||
.Returns(new byte[100]); // Dummy audio data
|
||
var service = new TtsService(mockCoquiWrapper.Object);
|
||
|
||
// Act
|
||
var result = await service.GenerateTtsAsync("Guten Morgen");
|
||
|
||
// Assert
|
||
Assert.NotNull(result);
|
||
Assert.NotEmpty(result);
|
||
}
|
||
}
|
||
```
|
||
|
||
##### **3. QuizService Tests**
|
||
|
||
```csharp
|
||
public class QuizServiceTests
|
||
{
|
||
[Fact]
|
||
public async Task SubmitQuiz_PassingScore_UpdatesProgress()
|
||
{
|
||
// Arrange
|
||
var options = new DbContextOptionsBuilder<AppDbContext>()
|
||
.UseInMemoryDatabase(databaseName: "TestDb")
|
||
.Options;
|
||
var dbContext = new AppDbContext(options);
|
||
|
||
var user = new User { Id = 1, CurrentLevel = "A1" };
|
||
var lesson = new Lesson { Id = 1, LevelId = 1 };
|
||
var quiz = new Quiz { Id = 1, LessonId = 1, PassingScore = 80 };
|
||
var question = new Question { Id = 1, QuizId = 1, CorrectAnswer = "der", Points = 1 };
|
||
|
||
dbContext.Users.Add(user);
|
||
dbContext.Lessons.Add(lesson);
|
||
dbContext.Quizzes.Add(quiz);
|
||
dbContext.Questions.Add(question);
|
||
await dbContext.SaveChangesAsync();
|
||
|
||
var service = new QuizService(dbContext);
|
||
var answers = new List<AnswerDto> { new AnswerDto { QuestionId = 1, UserAnswer = "der" } };
|
||
|
||
// Act
|
||
var result = await service.SubmitQuizAsync(1, 1, answers);
|
||
|
||
// Assert
|
||
Assert.True(result.Passed);
|
||
Assert.Equal(1, result.Score);
|
||
|
||
var userProgress = await dbContext.UserProgress.FirstOrDefaultAsync();
|
||
Assert.NotNull(userProgress);
|
||
Assert.True(userProgress.IsCompleted);
|
||
}
|
||
}
|
||
```
|
||
|
||
### **Frontend Tests (React)**
|
||
|
||
- **Framework**: Jest + React Testing Library.
|
||
- **Mocking**: Mock API calls and browser APIs (e.g., `navigator.mediaDevices`).
|
||
|
||
#### **Example Tests**
|
||
|
||
##### **1. AudioPlayer Tests**
|
||
|
||
```tsx
|
||
import { render, screen } from '@testing-library/react';
|
||
import { AudioPlayer } from './AudioPlayer';
|
||
|
||
test('renders audio player with correct src', () => {
|
||
render(<AudioPlayer audioUrl="/audio/test.wav" />);
|
||
const audioElement = screen.getByTestId('audio-player');
|
||
expect(audioElement).toHaveAttribute('src', '/audio/test.wav');
|
||
});
|
||
```
|
||
|
||
##### **2. Recorder Tests**
|
||
|
||
```tsx
|
||
import { render, screen, fireEvent, waitFor } from '@testing-library/react';
|
||
import { Recorder } from './Recorder';
|
||
|
||
test('calls onTranscript with recognized text', async () => {
|
||
const mockOnTranscript = jest.fn();
|
||
const mockMediaDevices = {
|
||
getUserMedia: jest.fn().mockResolvedValue({
|
||
getTracks: () => [],
|
||
stop: jest.fn(),
|
||
}),
|
||
};
|
||
Object.assign(navigator, { mediaDevices: mockMediaDevices });
|
||
|
||
render(<Recorder onTranscript={mockOnTranscript} />);
|
||
fireEvent.click(screen.getByText('🎤 Start Recording'));
|
||
|
||
// Mock fetch for /api/speech/recognize
|
||
global.fetch = jest.fn().mockResolvedValue({
|
||
json: () => Promise.resolve({ text: "Guten Morgen" }),
|
||
});
|
||
|
||
// Simulate stopping recording
|
||
await waitFor(() => {
|
||
expect(mockOnTranscript).toHaveBeenCalledWith("Guten Morgen");
|
||
});
|
||
});
|
||
```
|
||
|
||
##### **3. ListeningQuestion Tests**
|
||
|
||
```tsx
|
||
test('selects correct answer for listening question', async () => {
|
||
const mockOnAnswer = jest.fn();
|
||
render(
|
||
<ListeningQuestion
|
||
audioUrl="/audio/test.wav"
|
||
options={["A", "B"]}
|
||
correctAnswer="A"
|
||
onAnswer={mockOnAnswer}
|
||
/>
|
||
);
|
||
|
||
fireEvent.click(screen.getByText('▶️ Play'));
|
||
fireEvent.click(screen.getByText('A'));
|
||
|
||
await waitFor(() => {
|
||
expect(mockOnAnswer).toHaveBeenCalledWith(true);
|
||
});
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## **11. Development Roadmap**
|
||
|
||
### **Phase 1: MVP (A1 Only)**
|
||
|
||
|
||
| Task | Description | Priority | Estimated Time |
|
||
| ---- | -------------------------------------------------- | -------- | -------------- |
|
||
| 1.1 | Set up .NET 9.0 backend project | High | 2–4 hours |
|
||
| 1.2 | Set up PostgreSQL database | High | 1–2 hours |
|
||
| 1.3 | Implement Auth (JWT) | High | 4–6 hours |
|
||
| 1.4 | Design and implement database schema | High | 2–3 hours |
|
||
| 1.5 | Integrate Vosk for speech recognition | High | 4–8 hours |
|
||
| 1.6 | Integrate Coqui TTS for audio generation | High | 4–8 hours |
|
||
| 1.7 | Implement Lesson/Quiz/Story APIs | High | 6–10 hours |
|
||
| 1.8 | Set up React + TypeScript frontend | High | 2–4 hours |
|
||
| 1.9 | Implement Dashboard, Lesson, Quiz, and Story pages | High | 8–12 hours |
|
||
| 1.10 | Import vocabulary from Goethe/DW | High | 4–6 hours |
|
||
| 1.11 | Generate audio for vocabulary and story segments | High | 4–8 hours |
|
||
| 1.12 | Implement Mistral-Medium for story generation | Medium | 2–4 hours |
|
||
| 1.13 | Add listening comprehension to quizzes | Medium | 2–4 hours |
|
||
| 1.14 | Write unit tests for backend and frontend | Medium | 6–10 hours |
|
||
|
||
|
||
### **Phase 2: Polish**
|
||
|
||
|
||
| Task | Description | Priority | Estimated Time |
|
||
| ---- | ----------------------------------------------- | -------- | -------------- |
|
||
| 2.1 | Add gamification (points, badges, streaks) | Medium | 4–6 hours |
|
||
| 2.2 | Optimize for mobile devices | Medium | 4–8 hours |
|
||
| 2.3 | Add progress charts and analytics | Medium | 4–6 hours |
|
||
| 2.4 | Improve UI/UX (e.g., animations, accessibility) | Low | 4–8 hours |
|
||
|
||
|
||
### **Phase 3: Scaling**
|
||
|
||
|
||
| Task | Description | Priority | Estimated Time |
|
||
| ---- | -------------------------------------------------------- | -------- | -------------- |
|
||
| 3.1 | Add A2 level | Low | 8–12 hours |
|
||
| 3.2 | Add more interactive exercises (e.g., dialogue practice) | Low | 6–10 hours |
|
||
| 3.3 | Optimize performance (e.g., caching, lazy loading) | Low | 4–8 hours |
|
||
|
||
|
||
---
|
||
|
||
## **12. Example Workflows**
|
||
|
||
### **A. User Completes a Lesson**
|
||
|
||
1. **Dashboard**:
|
||
- User logs in and sees their dashboard with progress for A1.
|
||
- Clicks "Start Lesson 1: Daily Routine".
|
||
2. **Vocabulary Tab**:
|
||
- Sees 5 words: *aufstehen, frühstücken, Arbeit, schlafen, danke*.
|
||
- Each word has German, English, article, and an **audio button** (Coqui TTS).
|
||
3. **Grammar Tab**:
|
||
- Reads about present tense verbs (e.g., *ich stehe auf, du isst*).
|
||
- Listens to **audio examples** (Coqui TTS).
|
||
4. **Story Tab**:
|
||
- Reads: *"Anna steht um 7 Uhr auf. Sie frühstückt und geht zur Arbeit."*
|
||
- Listens to **audio** (Coqui TTS).
|
||
- Clicks on *aufstehen* to see the translation: "to get up".
|
||
5. **Practice Tab**:
|
||
- **Listening**: Plays audio of *"Anna schläft um 22 Uhr."* → selects the correct translation.
|
||
- **Speaking**: Records themselves saying *"Ich stehe um 8 Uhr auf."* → Vosk transcribes and validates.
|
||
- **Writing**: Types a sentence about their morning → Mistral-Medium provides feedback.
|
||
6. **Quiz Tab**:
|
||
- Answers 10 questions (MCQ, fill-in, listening).
|
||
- Scores 90% → lesson marked as completed; next lesson unlocked.
|
||
|
||
### **B. Admin Imports Vocabulary**
|
||
|
||
1. Admin calls `POST /api/vocabulary/import`.
|
||
2. Backend scrapes Goethe/DW and imports words into `Vocabulary` table.
|
||
3. For each word, Coqui TTS generates audio and stores the `AudioUrl`.
|
||
|
||
### **C. Story Generation**
|
||
|
||
1. Admin triggers `POST /api/ai/generate-story` for A1 Lesson 1.
|
||
2. Backend calls Mistral-Medium with the prompt:
|
||
```
|
||
Write a 2–3 paragraph story about daily life for a German A1 learner.
|
||
Use only A1 vocabulary and grammar. Include the words: ["aufstehen", "frühstücken"].
|
||
Theme: "A typical morning."
|
||
```
|
||
3. Mistral-Medium returns the story text.
|
||
4. Backend stores the story in `StorySegments` and generates audio using Coqui TTS.
|
||
|
||
---
|
||
|
||
## **13. Deployment Notes**
|
||
|
||
### **Self-Hosting Requirements**
|
||
|
||
1. **Server**:
|
||
- Linux/Windows server with Docker (recommended) or direct installation.
|
||
- Minimum **4GB RAM** (8GB recommended for Vosk/Coqui TTS).
|
||
- **Storage**: ~1GB for Vosk/Coqui models + audio files.
|
||
2. **Backend**:
|
||
- .NET 9.0 runtime.
|
||
- PostgreSQL database.
|
||
- Environment variables:
|
||
```bash
|
||
export ConnectionStrings__Default="Host=localhost;Database=germandb;Username=postgres;Password=yourpassword"
|
||
export Vosk__ModelPath="/models/vosk-model-de-0.22"
|
||
export Coqui__PythonPath="/usr/bin/python3"
|
||
export Coqui__ModelName="tts_models/de/deu/fairseq/vits"
|
||
export Mistral__ApiKey="your-mistral-api-key"
|
||
```
|
||
3. **Frontend**:
|
||
- Node.js (v18+) for Vite/React.
|
||
4. **Vosk**:
|
||
- Download `vosk-model-de-0.22` and place it in `/models/vosk`.
|
||
- Ensure the .NET app has read access to this directory.
|
||
- **Download Link**: [Vosk Models](https://alphacephei.com/vosk/models)
|
||
5. **Coqui TTS**:
|
||
- Install in a Python virtual environment:
|
||
6. **Mistral-Medium**:
|
||
- Sign up for an API key at [Mistral AI](https://mistral.ai/).
|
||
- Store the key in environment variables (as shown above).
|
||
|
||
### **Docker Setup (Optional)**
|
||
|
||
#### **1. Backend Dockerfile**
|
||
|
||
```dockerfile
|
||
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
|
||
WORKDIR /src
|
||
COPY . .
|
||
RUN dotnet publish -c Release -o /app
|
||
|
||
FROM mcr.microsoft.com/dotnet/aspnet:9.0
|
||
WORKDIR /app
|
||
COPY --from=build /app .
|
||
COPY /models/vosk-model-de-0.22 /models/vosk-model-de-0.22
|
||
ENV Vosk__ModelPath=/models/vosk-model-de-0.22
|
||
ENTRYPOINT ["dotnet", "GermanApp.dll"]
|
||
```
|
||
|
||
#### **2. Frontend Dockerfile**
|
||
|
||
```dockerfile
|
||
FROM node:18
|
||
WORKDIR /app
|
||
COPY package*.json ./
|
||
RUN npm install
|
||
COPY . .
|
||
RUN npm run build
|
||
EXPOSE 3000
|
||
CMD ["npm", "run", "dev"]
|
||
```
|
||
|
||
#### **3. PostgreSQL Docker Compose**
|
||
|
||
```yaml
|
||
version: '3'
|
||
services:
|
||
postgres:
|
||
image: postgres:15
|
||
environment:
|
||
POSTGRES_PASSWORD: yourpassword
|
||
POSTGRES_DB: germandb
|
||
ports:
|
||
- "5432:5432"
|
||
volumes:
|
||
- postgres_data:/var/lib/postgresql/data
|
||
|
||
backend:
|
||
build:
|
||
context: ./backend
|
||
dockerfile: Dockerfile
|
||
ports:
|
||
- "5000:80"
|
||
environment:
|
||
- ConnectionStrings__Default=Host=postgres;Database=germandb;Username=postgres;Password=yourpassword
|
||
- Vosk__ModelPath=/models/vosk-model-de-0.22
|
||
- Coqui__PythonPath=/usr/bin/python3
|
||
- Coqui__ModelName=tts_models/de/deu/fairseq/vits
|
||
- Mistral__ApiKey=${MISTRAL_API_KEY}
|
||
depends_on:
|
||
- postgres
|
||
volumes:
|
||
- ./models:/models
|
||
|
||
frontend:
|
||
build:
|
||
context: ./frontend
|
||
dockerfile: Dockerfile
|
||
ports:
|
||
- "3000:3000"
|
||
depends_on:
|
||
- backend
|
||
|
||
volumes:
|
||
postgres_data:
|
||
```
|
||
|
||
#### **4. Run the Stack**
|
||
|
||
```bash
|
||
# Create a .env file in the backend directory
|
||
echo "MISTRAL_API_KEY=your-api-key" > backend/.env
|
||
|
||
# Build and run the containers
|
||
docker-compose up --build
|
||
```
|
||
|
||
---
|
||
|
||
## **14. Open Questions**
|
||
|
||
1. **Coqui TTS Model**: Should we default to `tts_models/de/deu/fairseq/vits` (higher quality, ~1.5GB) or start with a smaller model (e.g., `tts_models/de/deu/tacotron2-DDC`) for faster generation?
|
||
2. **Audio Storage**: Should audio files be stored in:
|
||
- The filesystem (e.g., `/var/www/audio`)?
|
||
- A dedicated `AudioFiles` table with file paths?
|
||
- A CDN (e.g., AWS S3) for scalability?
|
||
3. **Vosk Confidence Threshold**: Should we reject transcriptions with <80% confidence for speaking exercises?
|
||
4. **Story Audio**: Should story segments have **both text and audio**, or just audio?
|
||
5. **Quiz Feedback**: Should users receive **immediate feedback** after each quiz question, or only at the end?
|
||
6. **Mobile App**: Should we consider wrapping the web app in a **mobile app** (e.g., using Capacitor) for better mobile UX?
|
||
|
||
---
|
||
|
||
## **Appendix: Glossary**
|
||
|
||
|
||
| Term | Definition |
|
||
| ------------------ | -------------------------------------------------------------------------- |
|
||
| **A1–C1** | Levels of the Common European Framework of Reference for Languages (CEFR). |
|
||
| **Vosk** | Open-source offline speech recognition toolkit. |
|
||
| **Coqui TTS** | Open-source text-to-speech library. |
|
||
| **Mistral-Medium** | Large language model by Mistral AI for text generation. |
|
||
| **TDD** | Test-Driven Development: Write tests before writing code. |
|
||
| **JWT** | JSON Web Token: Used for authentication. |
|
||
| **MCQ** | Multiple Choice Question. |
|
||
| **CEFR** | Common European Framework of Reference for Languages. |
|
||
|
||
|
||
---
|
||
|
||
## **Appendix: Sample Data**
|
||
|
||
### **A. Sample Vocabulary (A1)**
|
||
|
||
|
||
| Id | Word | Translation | Article | AudioUrl | Source |
|
||
| --- | ----------- | ---------------- | ------- | ----------------------- | ------ |
|
||
| 1 | aufstehen | to get up | - | /audio/vocabulary/1.wav | Goethe |
|
||
| 2 | frühstücken | to eat breakfast | - | /audio/vocabulary/2.wav | Goethe |
|
||
| 3 | Arbeit | work | die | /audio/vocabulary/3.wav | DW |
|
||
| 4 | Buch | book | das | /audio/vocabulary/4.wav | Goethe |
|
||
| 5 | schlafen | to sleep | - | /audio/vocabulary/5.wav | DW |
|
||
|
||
|
||
### **B. Sample Story Segment (A1)**
|
||
|
||
- **Id**: 1
|
||
- **LevelId**: 1 (A1)
|
||
- **LessonId**: 1
|
||
- **Content**:
|
||
```
|
||
Anna steht jeden Morgen um 7 Uhr auf. Sie geht ins Badezimmer und wäscht sich das Gesicht.
|
||
Dann frühstückt sie in der Küche. Sie isst ein Brot und trinkt einen Kaffee.
|
||
Um 8 Uhr geht Anna zur Arbeit. Sie arbeitet bis 17 Uhr. Am Abend kocht sie Abendessen und schläft um 22 Uhr.
|
||
```
|
||
- **AudioUrl**: `/audio/story/1.wav`
|
||
- **Order**: 1
|
||
|
||
### **C. Sample Quiz Questions (A1)**
|
||
|
||
|
||
| Id | QuizId | Type | Content | CorrectAnswer | AudioUrl | Options |
|
||
| --- | ------ | --------- | ----------------------------------- | --------------- | ----------------- | ------------------------------------- |
|
||
| 1 | 1 | mcq | What is the article for "Buch"? | das | - | ["der", "die", "das"] |
|
||
| 2 | 1 | fill_in | Anna _____ um 7 Uhr auf. | steht | - | - |
|
||
| 3 | 1 | listening | (Audio: "Das ist ein Buch.") | This is a book. | /audio/quiz/1.wav | ["This is a book.", "This is a cat."] |
|
||
| 4 | 1 | matching | Match the word to its article: Buch | das | - | ["der", "die", "das"] |
|
||
|
||
|
||
---
|
||
|
||
---
|
||
|
||
*Last updated: May 30, 2026*
|
||
*Author: Lasse Hansen (with assistance from Vibe, an AI by Mistral AI)* |