ai.interactive.fiction/CLIENT_TODO.md

Client Architecture Refactoring TODO List

Phase 1: Cleanup and Removal of Redundant Components

High Priority Removals

1. TTSPlayerModule - Complete Removal

   • Mark entire module for removal (completely redundant with SentenceQueueModule)
   • Identify all references to TTSPlayerModule across the codebase
   • Plan migration path for any UI components directly using TTSPlayerModule
   • Update any event listeners that expect TTSPlayerModule events

2. UIDisplayHandlerModule Cleanup

   • Remove pendingParagraphs queue
   • Remove processNextParagraph() method
   • Refactor displayText() to be a thin wrapper for SentenceQueue

3. KokoroTTSModule Cleanup

   • Remove pendingGenerations Map
   • Refactor generateSpeech() to work directly with SentenceQueue
   • Update iframe message handling to support the new architecture

4. LayoutRendererModule Cleanup

   • Remove TTS triggering from renderParagraph() method
   • Remove animation scheduling from rendering logic
   • Separate layout rendering from animation timing

Additional Obsolete Components

1. TextBufferModule Redundancies

   • Remove sentence preparation logic that will move to SentenceQueue
   • Remove any temporary sentence storage mechanisms
   • Identify and mark for removal any text processing that duplicates SentenceQueue functionality

2. AnimationQueueModule Overlaps

   • Identify animation scheduling that will be handled by SentenceQueue
   • Remove redundant animation timing calculations
   • Refactor to work as a service for SentenceQueue

3. TTSFactoryModule Redundancies

   • Remove any temporary audio storage that duplicates SentenceQueue functionality
   • Refactor speech generation methods to work with SentenceQueue
   • Streamline caching to avoid duplication with SentenceQueue

4. Event System Cleanup

   • Identify and remove redundant TTS-related events
   • Remove text animation events that will be handled by SentenceQueue
   • Consolidate playback state events

Phase 2: Enhanced SentenceQueueModule Implementation

Core Structure and Design

1. Design the Sentence Object Structure

   • Define comprehensive sentence object with fields for:
     • Unique ID
     • Original text
     • Processed text (hyphenated, typeset)
     • Layout information (breaks, nodes, typography)
     • Audio component (player, duration, data, type)
     • Status tracking (pending, processing, ready, playing, complete)

2. Implement Basic Queue Management

   • Create methods for adding sentences to the queue
   • Implement queue processing logic that maintains order
   • Add status tracking for each sentence in the queue
   • Implement priority handling for urgent sentences

Text Processing Integration

1. Integrate with Paragraph Layout

   • Connect to ParagraphLayoutModule for text processing
   • Implement hyphenation and typesetting in the queue
   • Store layout information in the sentence object
   • Ensure layout processing happens in parallel with audio

2. Text Animation Preparation

   • Calculate animation timing based on text length and settings
   • Prepare animation data for each word in the sentence
   • Store animation timing in the sentence object
   • Create animation player function for the sentence

Audio Processing Integration

1. TTS System Integration

   • Implement audio generation for Kokoro TTS
   • Implement browser TTS handling with duration estimation
   • Implement "none" TTS option with duration calculation
   • Create consistent player interface for all TTS types

2. Audio Data Management

   • Implement audio data storage in sentence objects
   • Connect with TTSFactoryModule's IndexedDB for persistent caching
   • Add audio preloading capabilities
   • Implement audio resource cleanup

Playback Coordination

1. Synchronized Playback

   • Implement coordinated text animation and audio playback
   • Create timing adjustment based on speed settings
   • Add event handling for playback states (start, pause, resume, complete)
   • Implement sentence transition handling

2. User Interaction Handling

   • Add support for fast-forwarding text/audio
   • Implement pause/resume functionality
   • Handle user interruptions gracefully
   • Support skipping to next sentence

Phase 3: Module Interface Updates

1. Update Module Interfaces

   • Create consistent interfaces for interacting with SentenceQueue
   • Update event system to work with sentence objects
   • Implement progress reporting for sentence processing
   • Add debugging and monitoring capabilities

2. Documentation and Examples

   • Document the new architecture and interfaces
   • Create usage examples for common scenarios
   • Update developer guidelines
   • Add migration guide for existing code

Phase 4: Testing and Validation

1. Unit Testing

   • Create tests for SentenceQueue core functionality
   • Test text processing integration
   • Test audio processing integration
   • Test playback coordination

2. Integration Testing

   • Test interaction between SentenceQueue and other modules
   • Validate timing and synchronization
   • Test error handling and recovery
   • Verify performance under load

3. User Experience Testing

   • Validate text animation quality
   • Test audio playback quality
   • Verify synchronization from user perspective
   • Test accessibility features

Implementation Strategy

1. Phased Rollout

   • Complete cleanup of redundant components
   • Implement SentenceQueue core structure
   • Add text processing integration
   • Add audio processing integration
   • Implement playback coordination
   • Gradually replace existing components

2. Backward Compatibility

   • Maintain support for existing interfaces during transition
   • Implement adapter patterns where needed
   • Add feature flags for enabling/disabling new architecture
   • Create fallback mechanisms for error recovery

3. Performance Optimization

   • Implement parallel processing where possible
   • Optimize memory usage for sentence objects
   • Add resource management for audio data
   • Implement efficient queue processing algorithms