Latest state before reworking with cluade 4.6.

2026-02-12 22:44:44 +00:00
parent b1387f4833
commit c745efd1d2
6 changed files with 565 additions and 570 deletions
@@ -0,0 +1,145 @@
+# Client Architecture Refactoring TODO List
+
+## Enhanced SentenceQueueModule Implementation
+
+### Phase 1: Core Structure and Design
+
+1. **Design the Sentence Object Structure**
+   - [ ] Define comprehensive sentence object with fields for:
+     - Unique ID
+     - Original text
+     - Processed text (hyphenated, typeset)
+     - Layout information (breaks, nodes, typography)
+     - Audio component (player, duration, data, type)
+     - Status tracking (pending, processing, ready, playing, complete)
+
+2. **Implement Basic Queue Management**
+   - [ ] Create methods for adding sentences to the queue
+   - [ ] Implement queue processing logic that maintains order
+   - [ ] Add status tracking for each sentence in the queue
+   - [ ] Implement priority handling for urgent sentences
+
+### Phase 2: Text Processing Integration
+
+1. **Integrate with Paragraph Layout**
+   - [ ] Connect to ParagraphLayoutModule for text processing
+   - [ ] Implement hyphenation and typesetting in the queue
+   - [ ] Store layout information in the sentence object
+   - [ ] Ensure layout processing happens in parallel with audio
+
+2. **Text Animation Preparation**
+   - [ ] Calculate animation timing based on text length and settings
+   - [ ] Prepare animation data for each word in the sentence
+   - [ ] Store animation timing in the sentence object
+   - [ ] Create animation player function for the sentence
+
+### Phase 3: Audio Processing Integration
+
+1. **TTS System Integration**
+   - [ ] Implement audio generation for Kokoro TTS
+   - [ ] Implement browser TTS handling with duration estimation
+   - [ ] Implement "none" TTS option with duration calculation
+   - [ ] Create consistent player interface for all TTS types
+
+2. **Audio Data Management**
+   - [ ] Implement audio data storage in sentence objects
+   - [ ] Connect with TTSFactoryModule's IndexedDB for persistent caching
+   - [ ] Add audio preloading capabilities
+   - [ ] Implement audio resource cleanup
+
+### Phase 4: Playback Coordination
+
+1. **Synchronized Playback**
+   - [ ] Implement coordinated text animation and audio playback
+   - [ ] Create timing adjustment based on speed settings
+   - [ ] Add event handling for playback states (start, pause, resume, complete)
+   - [ ] Implement sentence transition handling
+
+2. **User Interaction Handling**
+   - [ ] Add support for fast-forwarding text/audio
+   - [ ] Implement pause/resume functionality
+   - [ ] Handle user interruptions gracefully
+   - [ ] Support skipping to next sentence
+
+## Component Consolidation
+
+### Phase 1: Identify and Remove Redundancies
+
+1. **TTSPlayerModule Refactoring**
+   - [ ] Remove preloadedAudio Map (replaced by sentence objects)
+   - [ ] Remove preloadQueue (replaced by SentenceQueue)
+   - [ ] Update speak method to use SentenceQueue
+   - [ ] Refactor to be a thin wrapper around SentenceQueue
+
+2. **UIDisplayHandlerModule Refactoring**
+   - [ ] Remove pendingParagraphs queue (replaced by SentenceQueue)
+   - [ ] Update displayText to use SentenceQueue
+   - [ ] Modify processNextParagraph to work with sentence objects
+   - [ ] Update event handling to work with the new architecture
+
+3. **KokoroTTSModule Refactoring**
+   - [ ] Replace pendingGenerations Map with SentenceQueue integration
+   - [ ] Update generateSpeech to work with sentence objects
+   - [ ] Modify iframe communication to support the new structure
+   - [ ] Ensure backward compatibility during transition
+
+4. **TextBufferModule Refactoring**
+   - [ ] Move sentence preparation logic to SentenceQueue
+   - [ ] Update text handling to work with the new architecture
+   - [ ] Ensure proper integration with SentenceQueue
+   - [ ] Maintain high-level text management responsibilities
+
+### Phase 2: Interface Updates
+
+1. **Update Module Interfaces**
+   - [ ] Create consistent interfaces for interacting with SentenceQueue
+   - [ ] Update event system to work with sentence objects
+   - [ ] Implement progress reporting for sentence processing
+   - [ ] Add debugging and monitoring capabilities
+
+2. **Documentation and Examples**
+   - [ ] Document the new architecture and interfaces
+   - [ ] Create usage examples for common scenarios
+   - [ ] Update developer guidelines
+   - [ ] Add migration guide for existing code
+
+## Testing and Validation
+
+1. **Unit Testing**
+   - [ ] Create tests for SentenceQueue core functionality
+   - [ ] Test text processing integration
+   - [ ] Test audio processing integration
+   - [ ] Test playback coordination
+
+2. **Integration Testing**
+   - [ ] Test interaction between SentenceQueue and other modules
+   - [ ] Validate timing and synchronization
+   - [ ] Test error handling and recovery
+   - [ ] Verify performance under load
+
+3. **User Experience Testing**
+   - [ ] Validate text animation quality
+   - [ ] Test audio playback quality
+   - [ ] Verify synchronization from user perspective
+   - [ ] Test accessibility features
+
+## Implementation Strategy
+
+1. **Phased Rollout**
+   - [ ] Implement SentenceQueue core structure
+   - [ ] Add text processing integration
+   - [ ] Add audio processing integration
+   - [ ] Implement playback coordination
+   - [ ] Gradually replace existing components
+
+2. **Backward Compatibility**
+   - [ ] Maintain support for existing interfaces during transition
+   - [ ] Implement adapter patterns where needed
+   - [ ] Add feature flags for enabling/disabling new architecture
+   - [ ] Create fallback mechanisms for error recovery
+
+3. **Performance Optimization**
+   - [ ] Implement parallel processing where possible
+   - [ ] Optimize memory usage for sentence objects
+   - [ ] Add resource management for audio data
+   - [ ] Implement efficient queue processing algorithms