diff --git a/CLIENT_TODO.md b/CLIENT_TODO.md new file mode 100644 index 0000000..2fba969 --- /dev/null +++ b/CLIENT_TODO.md @@ -0,0 +1,165 @@ +# Client Architecture Refactoring TODO List + +## Phase 1: Cleanup and Removal of Redundant Components + +### High Priority Removals + +1. **TTSPlayerModule - Complete Removal** + - [X] Mark entire module for removal (completely redundant with SentenceQueueModule) + - [ ] Identify all references to TTSPlayerModule across the codebase + - [ ] Plan migration path for any UI components directly using TTSPlayerModule + - [ ] Update any event listeners that expect TTSPlayerModule events + +2. **UIDisplayHandlerModule Cleanup** + - [ ] Remove `pendingParagraphs` queue + - [ ] Remove `processNextParagraph()` method + - [ ] Refactor `displayText()` to be a thin wrapper for SentenceQueue + +3. **KokoroTTSModule Cleanup** + - [ ] Remove `pendingGenerations` Map + - [ ] Refactor `generateSpeech()` to work directly with SentenceQueue + - [ ] Update iframe message handling to support the new architecture + +4. **LayoutRendererModule Cleanup** + - [ ] Remove TTS triggering from `renderParagraph()` method + - [ ] Remove animation scheduling from rendering logic + - [ ] Separate layout rendering from animation timing + +### Additional Obsolete Components + +1. **TextBufferModule Redundancies** + - [ ] Remove sentence preparation logic that will move to SentenceQueue + - [ ] Remove any temporary sentence storage mechanisms + - [ ] Identify and mark for removal any text processing that duplicates SentenceQueue functionality + +2. **AnimationQueueModule Overlaps** + - [ ] Identify animation scheduling that will be handled by SentenceQueue + - [ ] Remove redundant animation timing calculations + - [ ] Refactor to work as a service for SentenceQueue + +3. **TTSFactoryModule Redundancies** + - [ ] Remove any temporary audio storage that duplicates SentenceQueue functionality + - [ ] Refactor speech generation methods to work with SentenceQueue + - [ ] Streamline caching to avoid duplication with SentenceQueue + +4. **Event System Cleanup** + - [ ] Identify and remove redundant TTS-related events + - [ ] Remove text animation events that will be handled by SentenceQueue + - [ ] Consolidate playback state events + +## Phase 2: Enhanced SentenceQueueModule Implementation + +### Core Structure and Design + +1. **Design the Sentence Object Structure** + - [ ] Define comprehensive sentence object with fields for: + - Unique ID + - Original text + - Processed text (hyphenated, typeset) + - Layout information (breaks, nodes, typography) + - Audio component (player, duration, data, type) + - Status tracking (pending, processing, ready, playing, complete) + +2. **Implement Basic Queue Management** + - [ ] Create methods for adding sentences to the queue + - [ ] Implement queue processing logic that maintains order + - [ ] Add status tracking for each sentence in the queue + - [ ] Implement priority handling for urgent sentences + +### Text Processing Integration + +1. **Integrate with Paragraph Layout** + - [ ] Connect to ParagraphLayoutModule for text processing + - [ ] Implement hyphenation and typesetting in the queue + - [ ] Store layout information in the sentence object + - [ ] Ensure layout processing happens in parallel with audio + +2. **Text Animation Preparation** + - [ ] Calculate animation timing based on text length and settings + - [ ] Prepare animation data for each word in the sentence + - [ ] Store animation timing in the sentence object + - [ ] Create animation player function for the sentence + +### Audio Processing Integration + +1. **TTS System Integration** + - [ ] Implement audio generation for Kokoro TTS + - [ ] Implement browser TTS handling with duration estimation + - [ ] Implement "none" TTS option with duration calculation + - [ ] Create consistent player interface for all TTS types + +2. **Audio Data Management** + - [ ] Implement audio data storage in sentence objects + - [ ] Connect with TTSFactoryModule's IndexedDB for persistent caching + - [ ] Add audio preloading capabilities + - [ ] Implement audio resource cleanup + +### Playback Coordination + +1. **Synchronized Playback** + - [ ] Implement coordinated text animation and audio playback + - [ ] Create timing adjustment based on speed settings + - [ ] Add event handling for playback states (start, pause, resume, complete) + - [ ] Implement sentence transition handling + +2. **User Interaction Handling** + - [ ] Add support for fast-forwarding text/audio + - [ ] Implement pause/resume functionality + - [ ] Handle user interruptions gracefully + - [ ] Support skipping to next sentence + +## Phase 3: Module Interface Updates + +1. **Update Module Interfaces** + - [ ] Create consistent interfaces for interacting with SentenceQueue + - [ ] Update event system to work with sentence objects + - [ ] Implement progress reporting for sentence processing + - [ ] Add debugging and monitoring capabilities + +2. **Documentation and Examples** + - [ ] Document the new architecture and interfaces + - [ ] Create usage examples for common scenarios + - [ ] Update developer guidelines + - [ ] Add migration guide for existing code + +## Phase 4: Testing and Validation + +1. **Unit Testing** + - [ ] Create tests for SentenceQueue core functionality + - [ ] Test text processing integration + - [ ] Test audio processing integration + - [ ] Test playback coordination + +2. **Integration Testing** + - [ ] Test interaction between SentenceQueue and other modules + - [ ] Validate timing and synchronization + - [ ] Test error handling and recovery + - [ ] Verify performance under load + +3. **User Experience Testing** + - [ ] Validate text animation quality + - [ ] Test audio playback quality + - [ ] Verify synchronization from user perspective + - [ ] Test accessibility features + +## Implementation Strategy + +1. **Phased Rollout** + - [ ] Complete cleanup of redundant components + - [ ] Implement SentenceQueue core structure + - [ ] Add text processing integration + - [ ] Add audio processing integration + - [ ] Implement playback coordination + - [ ] Gradually replace existing components + +2. **Backward Compatibility** + - [ ] Maintain support for existing interfaces during transition + - [ ] Implement adapter patterns where needed + - [ ] Add feature flags for enabling/disabling new architecture + - [ ] Create fallback mechanisms for error recovery + +3. **Performance Optimization** + - [ ] Implement parallel processing where possible + - [ ] Optimize memory usage for sentence objects + - [ ] Add resource management for audio data + - [ ] Implement efficient queue processing algorithms diff --git a/public/js/loader.js b/public/js/loader.js index e1ae211..086cdda 100644 --- a/public/js/loader.js +++ b/public/js/loader.js @@ -99,6 +99,7 @@ const ModuleLoader = (function() { { id: 'localization', script: '/js/localization-module.js', weight: 12 }, { id: 'text-processor', script: '/js/text-processor-module.js', weight: 15 }, { id: 'paragraph-layout', script: '/js/paragraph-layout-module.js', weight: 17 }, + { id: 'sentence-queue', script: '/js/sentence-queue-module.js', weight: 12 }, { id: 'layout-renderer', script: '/js/layout-renderer-module.js', weight: 13 }, // Add Layout Renderer module { id: 'animation-queue', script: '/js/animation-queue-module.js', weight: 12 }, @@ -109,7 +110,6 @@ const ModuleLoader = (function() { { id: 'elevenlabs', script: '/js/elevenlabs-tts-module.js', weight: 12 }, { id: 'openai', script: '/js/openai-tts-module.js', weight: 12 }, { id: 'tts-factory', script: '/js/tts-factory-module.js', weight: 13 }, // TTSFactory must be loaded before TTSPlayer - { id: 'tts-player', script: '/js/tts-player-module.js', weight: 13 }, // UI and interaction modules { id: 'text-buffer', script: '/js/text-buffer-module.js', weight: 12 }, diff --git a/public/js/sentence-queue-module.js b/public/js/sentence-queue-module.js new file mode 100644 index 0000000..a1828c5 --- /dev/null +++ b/public/js/sentence-queue-module.js @@ -0,0 +1,246 @@ +/** + * SentenceQueueModule + * Manages the preparation pipeline for sentences, including TTS generation + */ +import { BaseModule } from './base-module.js'; + +class SentenceQueueModule extends BaseModule { + constructor() { + super('sentence-queue', 'Sentence Queue'); + + // Dependencies + this.dependencies = ['text-buffer', 'tts-factory', 'tts-player']; + + // Queue state + this.sentenceQueue = []; + this.isProcessing = false; + this.onSentenceReadyCallback = null; + + // Bind methods + this.bindMethods([ + 'initialize', + 'addSentence', + 'processNextSentence', + 'setOnSentenceReady', + 'completeSentence' + ]); + } + + /** + * Initialize the module + * @returns {Promise} - Resolves with success status + */ + async initialize() { + try { + // Get dependencies + const textBuffer = this.getModule('text-buffer'); + + if (!textBuffer) { + console.error("SentenceQueue: TextBuffer dependency not found"); + return false; + } + + // Set up the text buffer to send sentences to this queue + textBuffer.setOnSentenceReady((sentence, callback) => { + this.addSentence(sentence, callback); + }); + + this.reportProgress(100, "Sentence queue ready"); + return true; + } catch (error) { + console.error("Error initializing Sentence Queue:", error); + return false; + } + } + + /** + * Set callback for when a sentence is ready for display + * @param {Function} callback - Function to call with prepared sentence + */ + setOnSentenceReady(callback) { + if (typeof callback === 'function') { + this.onSentenceReadyCallback = callback; + } + } + + /** + * Add a sentence to the queue + * @param {string} sentence - Sentence to add + * @param {Function} callback - Callback to call when sentence is processed + */ + addSentence(sentence, callback) { + this.sentenceQueue.push({ + text: sentence, + callback: callback + }); + + // Process the queue if not already processing + if (!this.isProcessing) { + this.processNextSentence(); + } + } + + /** + * Process the next sentence in the queue + */ + async processNextSentence() { + if (this.sentenceQueue.length === 0 || this.isProcessing) { + return; + } + + this.isProcessing = true; + const item = this.sentenceQueue[0]; // Don't remove yet + + try { + // Get TTS Factory + const ttsFactory = this.getModule('tts-factory'); + + if (!ttsFactory) { + console.error("SentenceQueue: TTSFactory dependency not found"); + this.completeSentence(item, { success: false, reason: 'no_tts_factory' }); + return; + } + + // Create a speech metadata object + const speechMetadata = await this.prepareSpeechMetadata(item.text); + + // If we have a callback for ready sentences, call it with the metadata + if (this.onSentenceReadyCallback) { + this.onSentenceReadyCallback(item.text, speechMetadata, () => { + // Remove from queue and process next + this.completeSentence(item, { success: true }); + }); + } else { + // No callback, just complete + this.completeSentence(item, { success: true }); + } + } catch (error) { + console.error("Error processing sentence:", error); + this.completeSentence(item, { success: false, reason: error.message }); + } + } + + /** + * Prepare speech metadata for a sentence + * @param {string} text - Text to prepare speech for + * @returns {Promise} - Speech metadata object + */ + async prepareSpeechMetadata(text) { + const ttsFactory = this.getModule('tts-factory'); + const ttsPlayer = this.getModule('tts-player'); + + if (!ttsFactory || !ttsPlayer) { + throw new Error("TTS dependencies not found"); + } + + // Check if TTS is enabled + const isTtsEnabled = ttsPlayer.isEnabled(); + + // If TTS is disabled, estimate duration based on character count + if (!isTtsEnabled) { + return this.estimateSpeechDuration(text); + } + + try { + // Preload the speech to get metadata + const result = await ttsFactory.preloadSpeech(text); + + if (!result.success) { + console.warn("SentenceQueue: Speech preload failed, using estimated duration"); + return this.estimateSpeechDuration(text); + } + + // Create a speech metadata object + return { + text: text, + duration: result.duration || this.estimateSpeechDuration(text).duration, + handler: ttsFactory.getActiveHandler() ? ttsFactory.getActiveHandler().id : null, + play: async () => { + return ttsFactory.speak(text); + }, + stop: () => { + return ttsFactory.stop(); + }, + isTtsEnabled: isTtsEnabled + }; + } catch (error) { + console.error("Error preparing speech metadata:", error); + return this.estimateSpeechDuration(text); + } + } + + /** + * Estimate speech duration based on character count + * @param {string} text - Text to estimate duration for + * @returns {Object} - Speech metadata object with estimated duration + */ + estimateSpeechDuration(text) { + // Average reading speed is about 14-15 characters per second + // We'll use a slightly slower rate for TTS + const charactersPerSecond = 12; + const ttsPlayer = this.getModule('tts-player'); + + // Get the current speed setting if available + let speedMultiplier = 1.0; + if (ttsPlayer) { + const ttsFactory = this.getModule('tts-factory'); + if (ttsFactory) { + // Get the current speed setting (typically 0.5-2.0) + const speed = ttsFactory.speed || 1.0; + speedMultiplier = speed; + } + } + + // Calculate estimated duration in milliseconds + const charCount = text.length; + const durationSeconds = charCount / (charactersPerSecond * speedMultiplier); + const durationMs = Math.max(durationSeconds * 1000, 500); // Minimum 500ms + + return { + text: text, + duration: durationMs, + handler: null, + play: async () => ({ success: false, reason: 'tts_disabled' }), + stop: () => true, + isTtsEnabled: false, + isEstimated: true + }; + } + + /** + * Complete processing of a sentence + * @param {Object} item - Queue item + * @param {Object} result - Processing result + */ + completeSentence(item, result) { + // Remove from queue + this.sentenceQueue.shift(); + + // Call the original callback + if (item.callback) { + item.callback(result); + } + + // Reset processing flag + this.isProcessing = false; + + // Process next sentence if any + if (this.sentenceQueue.length > 0) { + this.processNextSentence(); + } + } +} + +// Create the singleton instance +const SentenceQueue = new SentenceQueueModule(); + +// Export the module +export { SentenceQueue }; + +// Register with the module registry +if (window.moduleRegistry) { + window.moduleRegistry.register(SentenceQueue); +} + +// Keep a reference in window for loader system +window.SentenceQueue = SentenceQueue; diff --git a/public/js/tts-player-module.js b/public/js/tts-player-module.js deleted file mode 100644 index fb9b12f..0000000 --- a/public/js/tts-player-module.js +++ /dev/null @@ -1,364 +0,0 @@ -/** - * TTS Player Module - * Manages TTS functionality and interacts with available TTS handlers - */ -import { BaseModule } from './base-module.js'; - -class TTSPlayerModule extends BaseModule { - constructor() { - super('tts-player', 'TTS Player'); - - // Module dependencies - this.dependencies = ['tts-factory']; - - // TTS state - this.enabled = true; - this.currentSpeech = null; - this.pendingCallback = null; - - // Preloading mechanism - this.preloadQueue = []; - this.preloadedAudio = new Map(); // Cache for preloaded TTS - this.isPreloading = false; - - // Bind methods using parent's bindMethods utility - this.bindMethods([ - 'speak', - 'preloadSpeech', - 'processPreloadQueue', - 'stop', - 'enable', - 'isEnabled', - 'isSpeaking', - 'setVoice', - 'setSpeed', - 'getVoices', - 'toggle' - ]); - } - - /** - * Initialize the module - * @returns {Promise} - Resolves with success status - */ - async initialize() { - try { - this.reportProgress(20, "Initializing TTS Player"); - - // Get TTS Factory dependency - const ttsFactory = this.getModule('tts-factory'); - if (!ttsFactory) { - console.error("TTS Player: TTS Factory dependency not found"); - this.reportProgress(100, "TTS Player failed - missing dependencies"); - return false; - } - - // Check TTS availability from TTS Factory - this.enabled = ttsFactory.ttsAvailable && ttsFactory.getPreference('tts', 'enabled', false); - - // Set up event listeners - this.addEventListener(document, 'tts:enabled', (event) => { - if (event.detail) { - this.enabled = event.detail.enabled; - console.log(`TTS Player: TTS ${this.enabled ? 'enabled' : 'disabled'}`); - } - }); - - // Listen for TTS availability changes - this.addEventListener(document, 'tts:availability', (event) => { - if (event.detail) { - const available = event.detail.available; - console.log(`TTS Player: TTS availability changed to ${available ? 'available' : 'unavailable'}`); - - // If TTS becomes unavailable, disable it - if (!available) { - this.enabled = false; - // Notify UI that TTS is disabled - super.dispatchEvent('tts:stateChange', { - enabled: false, - available: false - }); - } - } - }); - - // Listen for TTS toggle events from UI - support both event names - this.addEventListener(document, 'tts:toggle', () => { - this.toggle(); - // Dispatch state change event for UI to update - super.dispatchEvent('tts:stateChange', { - enabled: this.enabled, - available: ttsFactory.ttsAvailable - }); - }); - - // Also listen for ui:tts:toggle events (from the main UI) - this.addEventListener(document, 'ui:tts:toggle', (event) => { - // If we have explicit enabled value, use it instead of toggling - if (event.detail && typeof event.detail.enabled === 'boolean') { - this.enabled = event.detail.enabled; - } else { - this.toggle(); - } - - // Dispatch state change event for UI to update - super.dispatchEvent('tts:stateChange', { - enabled: this.enabled, - available: ttsFactory.ttsAvailable - }); - }); - - // Request available TTS voices - this.reportProgress(60, "Checking for available TTS voices"); - const voices = await ttsFactory.getVoices(); - console.log(`TTS Player: ${voices.length} voices available`); - - this.reportProgress(100, "TTS Player ready"); - return true; - - } catch (error) { - console.error("Error initializing TTS Player:", error); - this.reportProgress(100, "TTS Player initialization failed"); - return false; - } - } - - /** - * Preload speech for a sentence - * @param {string} text - Text to preload - */ - preloadSpeech(text) { - if (!text || !this.enabled) return; - - // Skip if already preloaded or in queue - if (this.preloadedAudio.has(text) || this.preloadQueue.includes(text)) { - return; - } - - this.preloadQueue.push(text); - - // Start preloading if not already preloading and no active speech - if (!this.isPreloading && !this.currentSpeech) { - this.processPreloadQueue(); - } - } - - /** - * Process the preload queue - */ - processPreloadQueue() { - if (this.preloadQueue.length === 0 || this.isPreloading) { - return; - } - - this.isPreloading = true; - const text = this.preloadQueue.shift(); - - // Skip if already preloaded - if (this.preloadedAudio.has(text)) { - this.isPreloading = false; - this.processPreloadQueue(); - return; - } - - // Use TTS Factory to generate audio - const ttsFactory = this.getModule('tts-factory'); - if (ttsFactory) { - console.log(`Preloading TTS for: "${text.substring(0, 30)}${text.length > 30 ? '...' : ''}"`); - - ttsFactory.generateSpeech(text) - .then(audioData => { - if (audioData && audioData.success) { - this.preloadedAudio.set(text, audioData); - console.log(`TTS preloaded successfully for: "${text.substring(0, 30)}${text.length > 30 ? '...' : ''}"`); - } - }) - .catch(error => { - console.error("Error preloading TTS:", error); - }) - .finally(() => { - this.isPreloading = false; - - // Continue processing queue - if (this.preloadQueue.length > 0) { - this.processPreloadQueue(); - } - }); - } else { - this.isPreloading = false; - } - } - - /** - * Speak a sentence - * @param {string} text - Text to speak - * @param {Function} callback - Callback for when speech completes - * @returns {boolean} - Success status - */ - speak(text, callback = null) { - if (!this.enabled) { - if (callback) { - setTimeout(() => callback({ success: false, reason: 'disabled' }), 0); - } - return false; - } - - if (this.currentSpeech) { - // Stop current speech if any - this.stop(); - } - - this.currentSpeech = text; - this.pendingCallback = callback; - - const ttsFactory = this.getModule('tts-factory'); - if (ttsFactory) { - // Check if we have this preloaded - if (this.preloadedAudio.has(text)) { - const preloadedAudio = this.preloadedAudio.get(text); - this.preloadedAudio.delete(text); // Remove from cache after use - - console.log(`Using preloaded TTS for: "${text.substring(0, 30)}${text.length > 30 ? '...' : ''}"`); - - // Play the preloaded audio - ttsFactory.playAudio(preloadedAudio, (result) => { - // Store the completed result - this.currentSpeech = null; - - // Call the callback if provided - if (this.pendingCallback) { - this.pendingCallback(result); - this.pendingCallback = null; - } - - // Process next in preload queue if any - if (this.preloadQueue.length > 0 && !this.isPreloading) { - this.processPreloadQueue(); - } - }); - } else { - // Start TTS with regular speech if not preloaded - ttsFactory.speak(text, (result) => { - // Store the completed result - this.currentSpeech = null; - - // Call the callback if provided - if (this.pendingCallback) { - this.pendingCallback(result); - this.pendingCallback = null; - } - - // Process next in preload queue if any - if (this.preloadQueue.length > 0 && !this.isPreloading) { - this.processPreloadQueue(); - } - }); - } - - return true; - } else { - console.error("TTS Player: TTSFactory module not found in registry"); - if (callback) { - setTimeout(() => callback({ success: false, reason: 'no_tts_factory' }), 0); - } - return false; - } - } - - /** - * Stop speaking - */ - stop() { - const ttsFactory = this.getModule('tts-factory'); - if (ttsFactory) { - ttsFactory.stop(); - } - - this.currentSpeech = null; - this.pendingCallback = null; - } - - /** - * Toggle TTS enabled state - */ - toggle() { - this.enabled = !this.enabled; - this.enable(this.enabled); - return this.enabled; - } - - /** - * Enable or disable TTS - * @param {boolean} enabled - Whether TTS should be enabled - */ - enable(enabled) { - this.enabled = enabled; - console.log(`TTS Player: ${this.enabled ? 'Enabled' : 'Disabled'}`); - - // Save preference if persistence manager is available - const persistenceManager = this.getModule('persistence-manager'); - if (persistenceManager) { - persistenceManager.updatePreference('tts', 'enabled', this.enabled); - } - } - - /** - * Check if TTS is enabled - * @returns {boolean} - Whether TTS is enabled - */ - isEnabled() { - return this.enabled; - } - - /** - * Check if TTS is currently speaking - * @returns {boolean} - Whether TTS is speaking - */ - isSpeaking() { - const ttsFactory = this.getModule('tts-factory'); - if (ttsFactory) { - return ttsFactory.isSpeaking(); - } - return false; - } - - /** - * Set the voice to use - * @param {string} voice - Voice identifier - */ - setVoice(voice) { - const ttsFactory = this.getModule('tts-factory'); - if (ttsFactory) { - ttsFactory.configure({ voice }); - } - } - - /** - * Set the speech rate/speed - * @param {number} speed - Speech rate (0.5-2.0) - */ - setSpeed(speed) { - const ttsFactory = this.getModule('tts-factory'); - if (ttsFactory) { - ttsFactory.configure({ speed }); - } - } - - /** - * Get available voices - * @returns {Promise} - Resolves with array of voice objects - */ - async getVoices() { - const ttsFactory = this.getModule('tts-factory'); - if (ttsFactory) { - return ttsFactory.getVoices(); - } - return []; - } -} - -// Create the singleton instance -const TTSPlayer = new TTSPlayerModule(); - -// Export the module -export { TTSPlayer }; diff --git a/public/js/ui-display-handler-module.js b/public/js/ui-display-handler-module.js index d5b7a1c..69bbc5c 100644 --- a/public/js/ui-display-handler-module.js +++ b/public/js/ui-display-handler-module.js @@ -16,44 +16,21 @@ class UIDisplayHandlerModule extends BaseModule { this.pageLeft = null; this.pageRight = null; this.paragraphContainer = null; - - // State - this.currentParagraphId = 0; - this.pendingParagraphs = []; - + // Resources to preload this.cssPath = '/css/style.css'; this.imagesToPreload = [ '/images/book-3057904.png', '/images/brown-wooden-flooring.jpg' ]; - - // Configuration - this.updateConfig({ - typography: { - fontFamily: "'EB Garamond', serif", - fontSize: '1.15rem', - lineHeight: 1.5, - maxWidth: 600 - }, - animation: { - speed: 0.05, // Speed multiplier - useTypingAnimation: true - }, - display: { - showChoices: true - } - }); - + // Bind methods using parent's bindMethods utility this.bindMethods([ 'initializeContainers', 'displayText', - 'showChoices', - 'processNextParagraph', 'measureText', - 'updateTypographySettings', 'loadCSS', + 'showChoices', 'preloadImages' ]); @@ -75,38 +52,16 @@ class UIDisplayHandlerModule extends BaseModule { this.paragraphLayout = this.getModule('paragraph-layout'); this.layoutRenderer = this.getModule('layout-renderer'); this.animationQueue = this.getModule('animation-queue'); - - if (!this.paragraphLayout) { - console.error("UIDisplayHandler: Missing paragraph-layout module"); - return false; - } - - if (!this.layoutRenderer) { - console.error("UIDisplayHandler: Missing layout-renderer module"); - return false; - } - - if (!this.animationQueue) { - console.error("UIDisplayHandler: Missing animation-queue module"); - return false; - } - + this.reportProgress(50, "Initializing display containers"); // Initialize container elements this.initializeContainers(); this.reportProgress(70, "Setting up typography"); - - // Set up measure function for paragraph layout - const { fontSize, fontFamily } = this.config.typography; - this.paragraphLayout.updateFont(fontSize, fontFamily); - + this.reportProgress(90, "Setting up event listeners"); - - // Set up event listeners - this.setupEventListeners(); - + this.reportProgress(100, "UI Display Handler ready"); return true; } catch (error) { @@ -114,25 +69,7 @@ class UIDisplayHandlerModule extends BaseModule { return false; } } - - /** - * Set up event listeners - */ - setupEventListeners() { - // Listen for typography setting changes - this.addEventListener(document, 'ui:typography:update', (event) => { - if (event.detail) { - this.updateTypographySettings(event.detail); - } - }); - - // Listen for animation speed changes - this.addEventListener(document, 'ui:animation:speed', (event) => { - if (event.detail && typeof event.detail.speed === 'number') { - this.config.animation.speed = event.detail.speed; - } - }); - } + /** * Load CSS file asynchronously and wait for it to be applied @@ -141,16 +78,6 @@ class UIDisplayHandlerModule extends BaseModule { */ loadCSS(cssPath) { return new Promise((resolve, reject) => { - // Check if the stylesheet is already loaded - const existingLinks = document.querySelectorAll('link[rel="stylesheet"]'); - for (const link of existingLinks) { - if (link.href.includes(cssPath)) { - console.log(`UIDisplayHandler: CSS ${cssPath} already loaded`); - resolve(); - return; - } - } - // Create link element const link = document.createElement('link'); link.rel = 'stylesheet'; @@ -159,11 +86,7 @@ class UIDisplayHandlerModule extends BaseModule { // Set up load and error handlers link.onload = () => { console.log(`UIDisplayHandler: CSS ${cssPath} loaded successfully`); - - // Give a small delay for the CSS to be applied - setTimeout(() => { - resolve(); - }, 50); + resolve(); }; link.onerror = (error) => { @@ -347,49 +270,6 @@ class UIDisplayHandlerModule extends BaseModule { return this.context.measureText(text).width; } - /** - * Update typography settings - * @param {Object} settings - Typography settings - */ - updateTypographySettings(settings) { - let changed = false; - - if (settings.fontSize && settings.fontSize !== this.config.typography.fontSize) { - this.config.typography.fontSize = settings.fontSize; - changed = true; - } - - if (settings.fontFamily && settings.fontFamily !== this.config.typography.fontFamily) { - this.config.typography.fontFamily = settings.fontFamily; - changed = true; - } - - if (settings.lineHeight && settings.lineHeight !== this.config.typography.lineHeight) { - this.config.typography.lineHeight = settings.lineHeight; - changed = true; - } - - // If font settings changed, update the paragraph layout - if (changed && this.paragraphLayout) { - // Use the existing updateFont method - this.paragraphLayout.updateFont( - this.config.typography.fontSize, - this.config.typography.fontFamily - ); - - // Also update our local canvas context - if (this.context) { - this.context.font = `${this.config.typography.fontSize} ${this.config.typography.fontFamily}`; - } - - // Dispatch event about typography changes - this.dispatchEvent('ui:font:change', { - fontSize: this.config.typography.fontSize, - fontFamily: this.config.typography.fontFamily, - lineHeight: this.config.typography.lineHeight - }); - } - } /** * Display text in the UI @@ -421,83 +301,6 @@ class UIDisplayHandlerModule extends BaseModule { }); } - /** - * Process the next paragraph in the queue - */ - processNextParagraph() { - if (this.pendingParagraphs.length === 0) return; - - const paragraph = this.pendingParagraphs[0]; - const { id, text, options, resolve } = paragraph; - - try { - // Use the paragraph layout to calculate the optimal layout - const layout = this.paragraphLayout.calculateLayout(text, { - width: this.config.typography.maxWidth, - fontSize: options.fontSize || this.config.typography.fontSize, - fontFamily: options.fontFamily || this.config.typography.fontFamily, - lineHeight: options.lineHeight || this.config.typography.lineHeight - }); - - if (!layout) { - console.error("UIDisplayHandler: Failed to calculate paragraph layout"); - this.pendingParagraphs.shift(); // Remove this paragraph - resolve(null); - - // Process next paragraph if any - if (this.pendingParagraphs.length > 0) { - this.processNextParagraph(); - } - return; - } - - // Store the original text in the layout for TTS - layout.originalText = text; - - // Use the layout renderer to create the DOM elements - const paragraphElement = this.layoutRenderer.renderParagraph(layout, { - container: this.paragraphContainer, - id: id, - className: options.className || '', - style: options.style || {}, - animateWords: this.config.animation.useTypingAnimation, - animationSpeed: this.config.animation.speed, - tts: options.speak !== false, // Enable TTS by default - onComplete: () => { - // Dispatch event when paragraph is complete - this.dispatchEvent('ui:paragraph:complete', { id }); - - // Remove this paragraph from the queue - this.pendingParagraphs.shift(); - - // Resolve the promise with the paragraph element - resolve(paragraphElement); - - // Process next paragraph if any - if (this.pendingParagraphs.length > 0) { - this.processNextParagraph(); - } - } - }); - - // Scroll to the new paragraph - if (paragraphElement) { - paragraphElement.scrollIntoView({ behavior: 'smooth', block: 'end' }); - } - } catch (error) { - console.error("UIDisplayHandler: Error processing paragraph:", error); - - // Remove this paragraph from the queue - this.pendingParagraphs.shift(); - resolve(null); - - // Process next paragraph if any - if (this.pendingParagraphs.length > 0) { - this.processNextParagraph(); - } - } - } - /** * Show choices in the UI * @param {Array} choices - Array of choice objects diff --git a/references/CLIENT_TODO_v1.md b/references/CLIENT_TODO_v1.md new file mode 100644 index 0000000..730a564 --- /dev/null +++ b/references/CLIENT_TODO_v1.md @@ -0,0 +1,145 @@ +# Client Architecture Refactoring TODO List + +## Enhanced SentenceQueueModule Implementation + +### Phase 1: Core Structure and Design + +1. **Design the Sentence Object Structure** + - [ ] Define comprehensive sentence object with fields for: + - Unique ID + - Original text + - Processed text (hyphenated, typeset) + - Layout information (breaks, nodes, typography) + - Audio component (player, duration, data, type) + - Status tracking (pending, processing, ready, playing, complete) + +2. **Implement Basic Queue Management** + - [ ] Create methods for adding sentences to the queue + - [ ] Implement queue processing logic that maintains order + - [ ] Add status tracking for each sentence in the queue + - [ ] Implement priority handling for urgent sentences + +### Phase 2: Text Processing Integration + +1. **Integrate with Paragraph Layout** + - [ ] Connect to ParagraphLayoutModule for text processing + - [ ] Implement hyphenation and typesetting in the queue + - [ ] Store layout information in the sentence object + - [ ] Ensure layout processing happens in parallel with audio + +2. **Text Animation Preparation** + - [ ] Calculate animation timing based on text length and settings + - [ ] Prepare animation data for each word in the sentence + - [ ] Store animation timing in the sentence object + - [ ] Create animation player function for the sentence + +### Phase 3: Audio Processing Integration + +1. **TTS System Integration** + - [ ] Implement audio generation for Kokoro TTS + - [ ] Implement browser TTS handling with duration estimation + - [ ] Implement "none" TTS option with duration calculation + - [ ] Create consistent player interface for all TTS types + +2. **Audio Data Management** + - [ ] Implement audio data storage in sentence objects + - [ ] Connect with TTSFactoryModule's IndexedDB for persistent caching + - [ ] Add audio preloading capabilities + - [ ] Implement audio resource cleanup + +### Phase 4: Playback Coordination + +1. **Synchronized Playback** + - [ ] Implement coordinated text animation and audio playback + - [ ] Create timing adjustment based on speed settings + - [ ] Add event handling for playback states (start, pause, resume, complete) + - [ ] Implement sentence transition handling + +2. **User Interaction Handling** + - [ ] Add support for fast-forwarding text/audio + - [ ] Implement pause/resume functionality + - [ ] Handle user interruptions gracefully + - [ ] Support skipping to next sentence + +## Component Consolidation + +### Phase 1: Identify and Remove Redundancies + +1. **TTSPlayerModule Refactoring** + - [ ] Remove preloadedAudio Map (replaced by sentence objects) + - [ ] Remove preloadQueue (replaced by SentenceQueue) + - [ ] Update speak method to use SentenceQueue + - [ ] Refactor to be a thin wrapper around SentenceQueue + +2. **UIDisplayHandlerModule Refactoring** + - [ ] Remove pendingParagraphs queue (replaced by SentenceQueue) + - [ ] Update displayText to use SentenceQueue + - [ ] Modify processNextParagraph to work with sentence objects + - [ ] Update event handling to work with the new architecture + +3. **KokoroTTSModule Refactoring** + - [ ] Replace pendingGenerations Map with SentenceQueue integration + - [ ] Update generateSpeech to work with sentence objects + - [ ] Modify iframe communication to support the new structure + - [ ] Ensure backward compatibility during transition + +4. **TextBufferModule Refactoring** + - [ ] Move sentence preparation logic to SentenceQueue + - [ ] Update text handling to work with the new architecture + - [ ] Ensure proper integration with SentenceQueue + - [ ] Maintain high-level text management responsibilities + +### Phase 2: Interface Updates + +1. **Update Module Interfaces** + - [ ] Create consistent interfaces for interacting with SentenceQueue + - [ ] Update event system to work with sentence objects + - [ ] Implement progress reporting for sentence processing + - [ ] Add debugging and monitoring capabilities + +2. **Documentation and Examples** + - [ ] Document the new architecture and interfaces + - [ ] Create usage examples for common scenarios + - [ ] Update developer guidelines + - [ ] Add migration guide for existing code + +## Testing and Validation + +1. **Unit Testing** + - [ ] Create tests for SentenceQueue core functionality + - [ ] Test text processing integration + - [ ] Test audio processing integration + - [ ] Test playback coordination + +2. **Integration Testing** + - [ ] Test interaction between SentenceQueue and other modules + - [ ] Validate timing and synchronization + - [ ] Test error handling and recovery + - [ ] Verify performance under load + +3. **User Experience Testing** + - [ ] Validate text animation quality + - [ ] Test audio playback quality + - [ ] Verify synchronization from user perspective + - [ ] Test accessibility features + +## Implementation Strategy + +1. **Phased Rollout** + - [ ] Implement SentenceQueue core structure + - [ ] Add text processing integration + - [ ] Add audio processing integration + - [ ] Implement playback coordination + - [ ] Gradually replace existing components + +2. **Backward Compatibility** + - [ ] Maintain support for existing interfaces during transition + - [ ] Implement adapter patterns where needed + - [ ] Add feature flags for enabling/disabling new architecture + - [ ] Create fallback mechanisms for error recovery + +3. **Performance Optimization** + - [ ] Implement parallel processing where possible + - [ ] Optimize memory usage for sentence objects + - [ ] Add resource management for audio data + - [ ] Implement efficient queue processing algorithms \ No newline at end of file