Georg/ai.interactive.fiction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Latest state before reworking with cluade 4.6.

Browse Source
This commit is contained in:
Georg Tomitsch
2026-02-12 22:44:44 +00:00
parent b1387f4833
commit c745efd1d2
6 changed files with 565 additions and 570 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLIENT_TODO.md
+165
View File
@@ -0,0 +1,165 @@
# Client Architecture Refactoring TODO List
## Phase 1: Cleanup and Removal of Redundant Components
### High Priority Removals
1. **TTSPlayerModule - Complete Removal**
- [X] Mark entire module for removal (completely redundant with SentenceQueueModule)
- [ ] Identify all references to TTSPlayerModule across the codebase
- [ ] Plan migration path for any UI components directly using TTSPlayerModule
- [ ] Update any event listeners that expect TTSPlayerModule events
2. **UIDisplayHandlerModule Cleanup**
- [ ] Remove `pendingParagraphs` queue
- [ ] Remove `processNextParagraph()` method
- [ ] Refactor `displayText()` to be a thin wrapper for SentenceQueue
3. **KokoroTTSModule Cleanup**
- [ ] Remove `pendingGenerations` Map
- [ ] Refactor `generateSpeech()` to work directly with SentenceQueue
- [ ] Update iframe message handling to support the new architecture
4. **LayoutRendererModule Cleanup**
- [ ] Remove TTS triggering from `renderParagraph()` method
- [ ] Remove animation scheduling from rendering logic
- [ ] Separate layout rendering from animation timing
### Additional Obsolete Components
1. **TextBufferModule Redundancies**
- [ ] Remove sentence preparation logic that will move to SentenceQueue
- [ ] Remove any temporary sentence storage mechanisms
- [ ] Identify and mark for removal any text processing that duplicates SentenceQueue functionality
2. **AnimationQueueModule Overlaps**
- [ ] Identify animation scheduling that will be handled by SentenceQueue
- [ ] Remove redundant animation timing calculations
- [ ] Refactor to work as a service for SentenceQueue
3. **TTSFactoryModule Redundancies**
- [ ] Remove any temporary audio storage that duplicates SentenceQueue functionality
- [ ] Refactor speech generation methods to work with SentenceQueue
- [ ] Streamline caching to avoid duplication with SentenceQueue
4. **Event System Cleanup**
- [ ] Identify and remove redundant TTS-related events
- [ ] Remove text animation events that will be handled by SentenceQueue
- [ ] Consolidate playback state events
## Phase 2: Enhanced SentenceQueueModule Implementation
### Core Structure and Design
1. **Design the Sentence Object Structure**
- [ ] Define comprehensive sentence object with fields for:
- Unique ID
- Original text
- Processed text (hyphenated, typeset)
- Layout information (breaks, nodes, typography)
- Audio component (player, duration, data, type)
- Status tracking (pending, processing, ready, playing, complete)
2. **Implement Basic Queue Management**
- [ ] Create methods for adding sentences to the queue
- [ ] Implement queue processing logic that maintains order
- [ ] Add status tracking for each sentence in the queue
- [ ] Implement priority handling for urgent sentences
### Text Processing Integration
1. **Integrate with Paragraph Layout**
- [ ] Connect to ParagraphLayoutModule for text processing
- [ ] Implement hyphenation and typesetting in the queue
- [ ] Store layout information in the sentence object
- [ ] Ensure layout processing happens in parallel with audio
2. **Text Animation Preparation**
- [ ] Calculate animation timing based on text length and settings
- [ ] Prepare animation data for each word in the sentence
- [ ] Store animation timing in the sentence object
- [ ] Create animation player function for the sentence
### Audio Processing Integration
1. **TTS System Integration**
- [ ] Implement audio generation for Kokoro TTS
- [ ] Implement browser TTS handling with duration estimation
- [ ] Implement "none" TTS option with duration calculation
- [ ] Create consistent player interface for all TTS types
2. **Audio Data Management**
- [ ] Implement audio data storage in sentence objects
- [ ] Connect with TTSFactoryModule's IndexedDB for persistent caching
- [ ] Add audio preloading capabilities
- [ ] Implement audio resource cleanup
### Playback Coordination
1. **Synchronized Playback**
- [ ] Implement coordinated text animation and audio playback
- [ ] Create timing adjustment based on speed settings
- [ ] Add event handling for playback states (start, pause, resume, complete)
- [ ] Implement sentence transition handling
2. **User Interaction Handling**
- [ ] Add support for fast-forwarding text/audio
- [ ] Implement pause/resume functionality
- [ ] Handle user interruptions gracefully
- [ ] Support skipping to next sentence
## Phase 3: Module Interface Updates
1. **Update Module Interfaces**
- [ ] Create consistent interfaces for interacting with SentenceQueue
- [ ] Update event system to work with sentence objects
- [ ] Implement progress reporting for sentence processing
- [ ] Add debugging and monitoring capabilities
2. **Documentation and Examples**
- [ ] Document the new architecture and interfaces
- [ ] Create usage examples for common scenarios
- [ ] Update developer guidelines
- [ ] Add migration guide for existing code
## Phase 4: Testing and Validation
1. **Unit Testing**
- [ ] Create tests for SentenceQueue core functionality
- [ ] Test text processing integration
- [ ] Test audio processing integration
- [ ] Test playback coordination
2. **Integration Testing**
- [ ] Test interaction between SentenceQueue and other modules
- [ ] Validate timing and synchronization
- [ ] Test error handling and recovery
- [ ] Verify performance under load
3. **User Experience Testing**
- [ ] Validate text animation quality
- [ ] Test audio playback quality
- [ ] Verify synchronization from user perspective
- [ ] Test accessibility features
## Implementation Strategy
1. **Phased Rollout**
- [ ] Complete cleanup of redundant components
- [ ] Implement SentenceQueue core structure
- [ ] Add text processing integration
- [ ] Add audio processing integration
- [ ] Implement playback coordination
- [ ] Gradually replace existing components
2. **Backward Compatibility**
- [ ] Maintain support for existing interfaces during transition
- [ ] Implement adapter patterns where needed
- [ ] Add feature flags for enabling/disabling new architecture
- [ ] Create fallback mechanisms for error recovery
3. **Performance Optimization**
- [ ] Implement parallel processing where possible
- [ ] Optimize memory usage for sentence objects
- [ ] Add resource management for audio data
- [ ] Implement efficient queue processing algorithms
public/js/loader.js
+1 -1
View File
@@ -99,6 +99,7 @@ const ModuleLoader = (function() {
{ id: 'localization', script: '/js/localization-module.js', weight: 12 },
{ id: 'text-processor', script: '/js/text-processor-module.js', weight: 15 },
{ id: 'paragraph-layout', script: '/js/paragraph-layout-module.js', weight: 17 },
{ id: 'sentence-queue', script: '/js/sentence-queue-module.js', weight: 12 },
{ id: 'layout-renderer', script: '/js/layout-renderer-module.js', weight: 13 }, // Add Layout Renderer module
{ id: 'animation-queue', script: '/js/animation-queue-module.js', weight: 12 },
@@ -109,7 +110,6 @@ const ModuleLoader = (function() {
{ id: 'elevenlabs', script: '/js/elevenlabs-tts-module.js', weight: 12 },
{ id: 'openai', script: '/js/openai-tts-module.js', weight: 12 },
{ id: 'tts-factory', script: '/js/tts-factory-module.js', weight: 13 }, // TTSFactory must be loaded before TTSPlayer
{ id: 'tts-player', script: '/js/tts-player-module.js', weight: 13 },
// UI and interaction modules
{ id: 'text-buffer', script: '/js/text-buffer-module.js', weight: 12 },
public/js/sentence-queue-module.js
+246
View File
@@ -0,0 +1,246 @@
/**
* SentenceQueueModule
* Manages the preparation pipeline for sentences, including TTS generation
*/
import { BaseModule } from './base-module.js';
class SentenceQueueModule extends BaseModule {
constructor() {
super('sentence-queue', 'Sentence Queue');
// Dependencies
this.dependencies = ['text-buffer', 'tts-factory', 'tts-player'];
// Queue state
this.sentenceQueue = [];
this.isProcessing = false;
this.onSentenceReadyCallback = null;
// Bind methods
this.bindMethods([
'initialize',
'addSentence',
'processNextSentence',
'setOnSentenceReady',
'completeSentence'
]);
}
/**
* Initialize the module
* @returns {Promise<boolean>} - Resolves with success status
*/
async initialize() {
try {
// Get dependencies
const textBuffer = this.getModule('text-buffer');
if (!textBuffer) {
console.error("SentenceQueue: TextBuffer dependency not found");
return false;
}
// Set up the text buffer to send sentences to this queue
textBuffer.setOnSentenceReady((sentence, callback) => {
this.addSentence(sentence, callback);
});
this.reportProgress(100, "Sentence queue ready");
return true;
} catch (error) {
console.error("Error initializing Sentence Queue:", error);
return false;
}
}
/**
* Set callback for when a sentence is ready for display
* @param {Function} callback - Function to call with prepared sentence
*/
setOnSentenceReady(callback) {
if (typeof callback === 'function') {
this.onSentenceReadyCallback = callback;
}
}
/**
* Add a sentence to the queue
* @param {string} sentence - Sentence to add
* @param {Function} callback - Callback to call when sentence is processed
*/
addSentence(sentence, callback) {
this.sentenceQueue.push({
text: sentence,
callback: callback
});
// Process the queue if not already processing
if (!this.isProcessing) {
this.processNextSentence();
}
}
/**
* Process the next sentence in the queue
*/
async processNextSentence() {
if (this.sentenceQueue.length === 0 || this.isProcessing) {
return;
}
this.isProcessing = true;
const item = this.sentenceQueue[0]; // Don't remove yet
try {
// Get TTS Factory
const ttsFactory = this.getModule('tts-factory');
if (!ttsFactory) {
console.error("SentenceQueue: TTSFactory dependency not found");
this.completeSentence(item, { success: false, reason: 'no_tts_factory' });
return;
}
// Create a speech metadata object
const speechMetadata = await this.prepareSpeechMetadata(item.text);
// If we have a callback for ready sentences, call it with the metadata
if (this.onSentenceReadyCallback) {
this.onSentenceReadyCallback(item.text, speechMetadata, () => {
// Remove from queue and process next
this.completeSentence(item, { success: true });
});
} else {
// No callback, just complete
this.completeSentence(item, { success: true });
}
} catch (error) {
console.error("Error processing sentence:", error);
this.completeSentence(item, { success: false, reason: error.message });
}
}
/**
* Prepare speech metadata for a sentence
* @param {string} text - Text to prepare speech for
* @returns {Promise<Object>} - Speech metadata object
*/
async prepareSpeechMetadata(text) {
const ttsFactory = this.getModule('tts-factory');
const ttsPlayer = this.getModule('tts-player');
if (!ttsFactory || !ttsPlayer) {
throw new Error("TTS dependencies not found");
}
// Check if TTS is enabled
const isTtsEnabled = ttsPlayer.isEnabled();
// If TTS is disabled, estimate duration based on character count
if (!isTtsEnabled) {
return this.estimateSpeechDuration(text);
}
try {
// Preload the speech to get metadata
const result = await ttsFactory.preloadSpeech(text);
if (!result.success) {
console.warn("SentenceQueue: Speech preload failed, using estimated duration");
return this.estimateSpeechDuration(text);
}
// Create a speech metadata object
return {
text: text,
duration: result.duration || this.estimateSpeechDuration(text).duration,
handler: ttsFactory.getActiveHandler() ? ttsFactory.getActiveHandler().id : null,
play: async () => {
return ttsFactory.speak(text);
},
stop: () => {
return ttsFactory.stop();
},
isTtsEnabled: isTtsEnabled
};
} catch (error) {
console.error("Error preparing speech metadata:", error);
return this.estimateSpeechDuration(text);
}
}
/**
* Estimate speech duration based on character count
* @param {string} text - Text to estimate duration for
* @returns {Object} - Speech metadata object with estimated duration
*/
estimateSpeechDuration(text) {
// Average reading speed is about 14-15 characters per second
// We'll use a slightly slower rate for TTS
const charactersPerSecond = 12;
const ttsPlayer = this.getModule('tts-player');
// Get the current speed setting if available
let speedMultiplier = 1.0;
if (ttsPlayer) {
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
// Get the current speed setting (typically 0.5-2.0)
const speed = ttsFactory.speed || 1.0;
speedMultiplier = speed;
}
}
// Calculate estimated duration in milliseconds
const charCount = text.length;
const durationSeconds = charCount / (charactersPerSecond * speedMultiplier);
const durationMs = Math.max(durationSeconds * 1000, 500); // Minimum 500ms
return {
text: text,
duration: durationMs,
handler: null,
play: async () => ({ success: false, reason: 'tts_disabled' }),
stop: () => true,
isTtsEnabled: false,
isEstimated: true
};
}
/**
* Complete processing of a sentence
* @param {Object} item - Queue item
* @param {Object} result - Processing result
*/
completeSentence(item, result) {
// Remove from queue
this.sentenceQueue.shift();
// Call the original callback
if (item.callback) {
item.callback(result);
}
// Reset processing flag
this.isProcessing = false;
// Process next sentence if any
if (this.sentenceQueue.length > 0) {
this.processNextSentence();
}
}
}
// Create the singleton instance
const SentenceQueue = new SentenceQueueModule();
// Export the module
export { SentenceQueue };
// Register with the module registry
if (window.moduleRegistry) {
window.moduleRegistry.register(SentenceQueue);
}
// Keep a reference in window for loader system
window.SentenceQueue = SentenceQueue;
public/js/tts-player-module.js
-364
View File
@@ -1,364 +0,0 @@
/**
* TTS Player Module
* Manages TTS functionality and interacts with available TTS handlers
*/
import { BaseModule } from './base-module.js';
class TTSPlayerModule extends BaseModule {
constructor() {
super('tts-player', 'TTS Player');
// Module dependencies
this.dependencies = ['tts-factory'];
// TTS state
this.enabled = true;
this.currentSpeech = null;
this.pendingCallback = null;
// Preloading mechanism
this.preloadQueue = [];
this.preloadedAudio = new Map(); // Cache for preloaded TTS
this.isPreloading = false;
// Bind methods using parent's bindMethods utility
this.bindMethods([
'speak',
'preloadSpeech',
'processPreloadQueue',
'stop',
'enable',
'isEnabled',
'isSpeaking',
'setVoice',
'setSpeed',
'getVoices',
'toggle'
]);
}
/**
* Initialize the module
* @returns {Promise<boolean>} - Resolves with success status
*/
async initialize() {
try {
this.reportProgress(20, "Initializing TTS Player");
// Get TTS Factory dependency
const ttsFactory = this.getModule('tts-factory');
if (!ttsFactory) {
console.error("TTS Player: TTS Factory dependency not found");
this.reportProgress(100, "TTS Player failed - missing dependencies");
return false;
}
// Check TTS availability from TTS Factory
this.enabled = ttsFactory.ttsAvailable && ttsFactory.getPreference('tts', 'enabled', false);
// Set up event listeners
this.addEventListener(document, 'tts:enabled', (event) => {
if (event.detail) {
this.enabled = event.detail.enabled;
console.log(`TTS Player: TTS ${this.enabled ? 'enabled' : 'disabled'}`);
}
});
// Listen for TTS availability changes
this.addEventListener(document, 'tts:availability', (event) => {
if (event.detail) {
const available = event.detail.available;
console.log(`TTS Player: TTS availability changed to ${available ? 'available' : 'unavailable'}`);
// If TTS becomes unavailable, disable it
if (!available) {
this.enabled = false;
// Notify UI that TTS is disabled
super.dispatchEvent('tts:stateChange', {
enabled: false,
available: false
});
}
}
});
// Listen for TTS toggle events from UI - support both event names
this.addEventListener(document, 'tts:toggle', () => {
this.toggle();
// Dispatch state change event for UI to update
super.dispatchEvent('tts:stateChange', {
enabled: this.enabled,
available: ttsFactory.ttsAvailable
});
});
// Also listen for ui:tts:toggle events (from the main UI)
this.addEventListener(document, 'ui:tts:toggle', (event) => {
// If we have explicit enabled value, use it instead of toggling
if (event.detail && typeof event.detail.enabled === 'boolean') {
this.enabled = event.detail.enabled;
} else {
this.toggle();
}
// Dispatch state change event for UI to update
super.dispatchEvent('tts:stateChange', {
enabled: this.enabled,
available: ttsFactory.ttsAvailable
});
});
// Request available TTS voices
this.reportProgress(60, "Checking for available TTS voices");
const voices = await ttsFactory.getVoices();
console.log(`TTS Player: ${voices.length} voices available`);
this.reportProgress(100, "TTS Player ready");
return true;
} catch (error) {
console.error("Error initializing TTS Player:", error);
this.reportProgress(100, "TTS Player initialization failed");
return false;
}
}
/**
* Preload speech for a sentence
* @param {string} text - Text to preload
*/
preloadSpeech(text) {
if (!text || !this.enabled) return;
// Skip if already preloaded or in queue
if (this.preloadedAudio.has(text) || this.preloadQueue.includes(text)) {
return;
}
this.preloadQueue.push(text);
// Start preloading if not already preloading and no active speech
if (!this.isPreloading && !this.currentSpeech) {
this.processPreloadQueue();
}
}
/**
* Process the preload queue
*/
processPreloadQueue() {
if (this.preloadQueue.length === 0 || this.isPreloading) {
return;
}
this.isPreloading = true;
const text = this.preloadQueue.shift();
// Skip if already preloaded
if (this.preloadedAudio.has(text)) {
this.isPreloading = false;
this.processPreloadQueue();
return;
}
// Use TTS Factory to generate audio
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
console.log(`Preloading TTS for: "${text.substring(0, 30)}${text.length > 30 ? '...' : ''}"`);
ttsFactory.generateSpeech(text)
.then(audioData => {
if (audioData && audioData.success) {
this.preloadedAudio.set(text, audioData);
console.log(`TTS preloaded successfully for: "${text.substring(0, 30)}${text.length > 30 ? '...' : ''}"`);
}
})
.catch(error => {
console.error("Error preloading TTS:", error);
})
.finally(() => {
this.isPreloading = false;
// Continue processing queue
if (this.preloadQueue.length > 0) {
this.processPreloadQueue();
}
});
} else {
this.isPreloading = false;
}
}
/**
* Speak a sentence
* @param {string} text - Text to speak
* @param {Function} callback - Callback for when speech completes
* @returns {boolean} - Success status
*/
speak(text, callback = null) {
if (!this.enabled) {
if (callback) {
setTimeout(() => callback({ success: false, reason: 'disabled' }), 0);
}
return false;
}
if (this.currentSpeech) {
// Stop current speech if any
this.stop();
}
this.currentSpeech = text;
this.pendingCallback = callback;
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
// Check if we have this preloaded
if (this.preloadedAudio.has(text)) {
const preloadedAudio = this.preloadedAudio.get(text);
this.preloadedAudio.delete(text); // Remove from cache after use
console.log(`Using preloaded TTS for: "${text.substring(0, 30)}${text.length > 30 ? '...' : ''}"`);
// Play the preloaded audio
ttsFactory.playAudio(preloadedAudio, (result) => {
// Store the completed result
this.currentSpeech = null;
// Call the callback if provided
if (this.pendingCallback) {
this.pendingCallback(result);
this.pendingCallback = null;
}
// Process next in preload queue if any
if (this.preloadQueue.length > 0 && !this.isPreloading) {
this.processPreloadQueue();
}
});
} else {
// Start TTS with regular speech if not preloaded
ttsFactory.speak(text, (result) => {
// Store the completed result
this.currentSpeech = null;
// Call the callback if provided
if (this.pendingCallback) {
this.pendingCallback(result);
this.pendingCallback = null;
}
// Process next in preload queue if any
if (this.preloadQueue.length > 0 && !this.isPreloading) {
this.processPreloadQueue();
}
});
}
return true;
} else {
console.error("TTS Player: TTSFactory module not found in registry");
if (callback) {
setTimeout(() => callback({ success: false, reason: 'no_tts_factory' }), 0);
}
return false;
}
}
/**
* Stop speaking
*/
stop() {
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
ttsFactory.stop();
}
this.currentSpeech = null;
this.pendingCallback = null;
}
/**
* Toggle TTS enabled state
*/
toggle() {
this.enabled = !this.enabled;
this.enable(this.enabled);
return this.enabled;
}
/**
* Enable or disable TTS
* @param {boolean} enabled - Whether TTS should be enabled
*/
enable(enabled) {
this.enabled = enabled;
console.log(`TTS Player: ${this.enabled ? 'Enabled' : 'Disabled'}`);
// Save preference if persistence manager is available
const persistenceManager = this.getModule('persistence-manager');
if (persistenceManager) {
persistenceManager.updatePreference('tts', 'enabled', this.enabled);
}
}
/**
* Check if TTS is enabled
* @returns {boolean} - Whether TTS is enabled
*/
isEnabled() {
return this.enabled;
}
/**
* Check if TTS is currently speaking
* @returns {boolean} - Whether TTS is speaking
*/
isSpeaking() {
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
return ttsFactory.isSpeaking();
}
return false;
}
/**
* Set the voice to use
* @param {string} voice - Voice identifier
*/
setVoice(voice) {
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
ttsFactory.configure({ voice });
}
}
/**
* Set the speech rate/speed
* @param {number} speed - Speech rate (0.5-2.0)
*/
setSpeed(speed) {
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
ttsFactory.configure({ speed });
}
}
/**
* Get available voices
* @returns {Promise<Array>} - Resolves with array of voice objects
*/
async getVoices() {
const ttsFactory = this.getModule('tts-factory');
if (ttsFactory) {
return ttsFactory.getVoices();
}
return [];
}
}
// Create the singleton instance
const TTSPlayer = new TTSPlayerModule();
// Export the module
export { TTSPlayer };
public/js/ui-display-handler-module.js
+1 -198
View File
@@ -17,10 +17,6 @@ class UIDisplayHandlerModule extends BaseModule {
this.pageRight = null;
this.paragraphContainer = null;
// State
this.currentParagraphId = 0;
this.pendingParagraphs = [];
// Resources to preload
this.cssPath = '/css/style.css';
this.imagesToPreload = [
@@ -28,32 +24,13 @@ class UIDisplayHandlerModule extends BaseModule {
'/images/brown-wooden-flooring.jpg'
];
// Configuration
this.updateConfig({
typography: {
fontFamily: "'EB Garamond', serif",
fontSize: '1.15rem',
lineHeight: 1.5,
maxWidth: 600
},
animation: {
speed: 0.05, // Speed multiplier
useTypingAnimation: true
},
display: {
showChoices: true
}
});
// Bind methods using parent's bindMethods utility
this.bindMethods([
'initializeContainers',
'displayText',
'showChoices',
'processNextParagraph',
'measureText',
'updateTypographySettings',
'loadCSS',
'showChoices',
'preloadImages'
]);
@@ -76,21 +53,6 @@ class UIDisplayHandlerModule extends BaseModule {
this.layoutRenderer = this.getModule('layout-renderer');
this.animationQueue = this.getModule('animation-queue');
if (!this.paragraphLayout) {
console.error("UIDisplayHandler: Missing paragraph-layout module");
return false;
}
if (!this.layoutRenderer) {
console.error("UIDisplayHandler: Missing layout-renderer module");
return false;
}
if (!this.animationQueue) {
console.error("UIDisplayHandler: Missing animation-queue module");
return false;
}
this.reportProgress(50, "Initializing display containers");
// Initialize container elements
@@ -98,15 +60,8 @@ class UIDisplayHandlerModule extends BaseModule {
this.reportProgress(70, "Setting up typography");
// Set up measure function for paragraph layout
const { fontSize, fontFamily } = this.config.typography;
this.paragraphLayout.updateFont(fontSize, fontFamily);
this.reportProgress(90, "Setting up event listeners");
// Set up event listeners
this.setupEventListeners();
this.reportProgress(100, "UI Display Handler ready");
return true;
} catch (error) {
@@ -115,24 +70,6 @@ class UIDisplayHandlerModule extends BaseModule {
}
}
/**
* Set up event listeners
*/
setupEventListeners() {
// Listen for typography setting changes
this.addEventListener(document, 'ui:typography:update', (event) => {
if (event.detail) {
this.updateTypographySettings(event.detail);
}
});
// Listen for animation speed changes
this.addEventListener(document, 'ui:animation:speed', (event) => {
if (event.detail && typeof event.detail.speed === 'number') {
this.config.animation.speed = event.detail.speed;
}
});
}
/**
* Load CSS file asynchronously and wait for it to be applied
@@ -141,16 +78,6 @@ class UIDisplayHandlerModule extends BaseModule {
*/
loadCSS(cssPath) {
return new Promise((resolve, reject) => {
// Check if the stylesheet is already loaded
const existingLinks = document.querySelectorAll('link[rel="stylesheet"]');
for (const link of existingLinks) {
if (link.href.includes(cssPath)) {
console.log(`UIDisplayHandler: CSS ${cssPath} already loaded`);
resolve();
return;
}
}
// Create link element
const link = document.createElement('link');
link.rel = 'stylesheet';
@@ -159,11 +86,7 @@ class UIDisplayHandlerModule extends BaseModule {
// Set up load and error handlers
link.onload = () => {
console.log(`UIDisplayHandler: CSS ${cssPath} loaded successfully`);
// Give a small delay for the CSS to be applied
setTimeout(() => {
resolve();
}, 50);
};
link.onerror = (error) => {
@@ -347,49 +270,6 @@ class UIDisplayHandlerModule extends BaseModule {
return this.context.measureText(text).width;
}
/**
* Update typography settings
* @param {Object} settings - Typography settings
*/
updateTypographySettings(settings) {
let changed = false;
if (settings.fontSize && settings.fontSize !== this.config.typography.fontSize) {
this.config.typography.fontSize = settings.fontSize;
changed = true;
}
if (settings.fontFamily && settings.fontFamily !== this.config.typography.fontFamily) {
this.config.typography.fontFamily = settings.fontFamily;
changed = true;
}
if (settings.lineHeight && settings.lineHeight !== this.config.typography.lineHeight) {
this.config.typography.lineHeight = settings.lineHeight;
changed = true;
}
// If font settings changed, update the paragraph layout
if (changed && this.paragraphLayout) {
// Use the existing updateFont method
this.paragraphLayout.updateFont(
this.config.typography.fontSize,
this.config.typography.fontFamily
);
// Also update our local canvas context
if (this.context) {
this.context.font = `${this.config.typography.fontSize} ${this.config.typography.fontFamily}`;
}
// Dispatch event about typography changes
this.dispatchEvent('ui:font:change', {
fontSize: this.config.typography.fontSize,
fontFamily: this.config.typography.fontFamily,
lineHeight: this.config.typography.lineHeight
});
}
}
/**
* Display text in the UI
@@ -421,83 +301,6 @@ class UIDisplayHandlerModule extends BaseModule {
});
}
/**
* Process the next paragraph in the queue
*/
processNextParagraph() {
if (this.pendingParagraphs.length === 0) return;
const paragraph = this.pendingParagraphs[0];
const { id, text, options, resolve } = paragraph;
try {
// Use the paragraph layout to calculate the optimal layout
const layout = this.paragraphLayout.calculateLayout(text, {
width: this.config.typography.maxWidth,
fontSize: options.fontSize || this.config.typography.fontSize,
fontFamily: options.fontFamily || this.config.typography.fontFamily,
lineHeight: options.lineHeight || this.config.typography.lineHeight
});
if (!layout) {
console.error("UIDisplayHandler: Failed to calculate paragraph layout");
this.pendingParagraphs.shift(); // Remove this paragraph
resolve(null);
// Process next paragraph if any
if (this.pendingParagraphs.length > 0) {
this.processNextParagraph();
}
return;
}
// Store the original text in the layout for TTS
layout.originalText = text;
// Use the layout renderer to create the DOM elements
const paragraphElement = this.layoutRenderer.renderParagraph(layout, {
container: this.paragraphContainer,
id: id,
className: options.className || '',
style: options.style || {},
animateWords: this.config.animation.useTypingAnimation,
animationSpeed: this.config.animation.speed,
tts: options.speak !== false, // Enable TTS by default
onComplete: () => {
// Dispatch event when paragraph is complete
this.dispatchEvent('ui:paragraph:complete', { id });
// Remove this paragraph from the queue
this.pendingParagraphs.shift();
// Resolve the promise with the paragraph element
resolve(paragraphElement);
// Process next paragraph if any
if (this.pendingParagraphs.length > 0) {
this.processNextParagraph();
}
}
});
// Scroll to the new paragraph
if (paragraphElement) {
paragraphElement.scrollIntoView({ behavior: 'smooth', block: 'end' });
}
} catch (error) {
console.error("UIDisplayHandler: Error processing paragraph:", error);
// Remove this paragraph from the queue
this.pendingParagraphs.shift();
resolve(null);
// Process next paragraph if any
if (this.pendingParagraphs.length > 0) {
this.processNextParagraph();
}
}
}
/**
* Show choices in the UI
* @param {Array<Object>} choices - Array of choice objects
references/CLIENT_TODO_v1.md
+145
View File
@@ -0,0 +1,145 @@
# Client Architecture Refactoring TODO List
## Enhanced SentenceQueueModule Implementation
### Phase 1: Core Structure and Design
1. **Design the Sentence Object Structure**
- [ ] Define comprehensive sentence object with fields for:
- Unique ID
- Original text
- Processed text (hyphenated, typeset)
- Layout information (breaks, nodes, typography)
- Audio component (player, duration, data, type)
- Status tracking (pending, processing, ready, playing, complete)
2. **Implement Basic Queue Management**
- [ ] Create methods for adding sentences to the queue
- [ ] Implement queue processing logic that maintains order
- [ ] Add status tracking for each sentence in the queue
- [ ] Implement priority handling for urgent sentences
### Phase 2: Text Processing Integration
1. **Integrate with Paragraph Layout**
- [ ] Connect to ParagraphLayoutModule for text processing
- [ ] Implement hyphenation and typesetting in the queue
- [ ] Store layout information in the sentence object
- [ ] Ensure layout processing happens in parallel with audio
2. **Text Animation Preparation**
- [ ] Calculate animation timing based on text length and settings
- [ ] Prepare animation data for each word in the sentence
- [ ] Store animation timing in the sentence object
- [ ] Create animation player function for the sentence
### Phase 3: Audio Processing Integration
1. **TTS System Integration**
- [ ] Implement audio generation for Kokoro TTS
- [ ] Implement browser TTS handling with duration estimation
- [ ] Implement "none" TTS option with duration calculation
- [ ] Create consistent player interface for all TTS types
2. **Audio Data Management**
- [ ] Implement audio data storage in sentence objects
- [ ] Connect with TTSFactoryModule's IndexedDB for persistent caching
- [ ] Add audio preloading capabilities
- [ ] Implement audio resource cleanup
### Phase 4: Playback Coordination
1. **Synchronized Playback**
- [ ] Implement coordinated text animation and audio playback
- [ ] Create timing adjustment based on speed settings
- [ ] Add event handling for playback states (start, pause, resume, complete)
- [ ] Implement sentence transition handling
2. **User Interaction Handling**
- [ ] Add support for fast-forwarding text/audio
- [ ] Implement pause/resume functionality
- [ ] Handle user interruptions gracefully
- [ ] Support skipping to next sentence
## Component Consolidation
### Phase 1: Identify and Remove Redundancies
1. **TTSPlayerModule Refactoring**
- [ ] Remove preloadedAudio Map (replaced by sentence objects)
- [ ] Remove preloadQueue (replaced by SentenceQueue)
- [ ] Update speak method to use SentenceQueue
- [ ] Refactor to be a thin wrapper around SentenceQueue
2. **UIDisplayHandlerModule Refactoring**
- [ ] Remove pendingParagraphs queue (replaced by SentenceQueue)
- [ ] Update displayText to use SentenceQueue
- [ ] Modify processNextParagraph to work with sentence objects
- [ ] Update event handling to work with the new architecture
3. **KokoroTTSModule Refactoring**
- [ ] Replace pendingGenerations Map with SentenceQueue integration
- [ ] Update generateSpeech to work with sentence objects
- [ ] Modify iframe communication to support the new structure
- [ ] Ensure backward compatibility during transition
4. **TextBufferModule Refactoring**
- [ ] Move sentence preparation logic to SentenceQueue
- [ ] Update text handling to work with the new architecture
- [ ] Ensure proper integration with SentenceQueue
- [ ] Maintain high-level text management responsibilities
### Phase 2: Interface Updates
1. **Update Module Interfaces**
- [ ] Create consistent interfaces for interacting with SentenceQueue
- [ ] Update event system to work with sentence objects
- [ ] Implement progress reporting for sentence processing
- [ ] Add debugging and monitoring capabilities
2. **Documentation and Examples**
- [ ] Document the new architecture and interfaces
- [ ] Create usage examples for common scenarios
- [ ] Update developer guidelines
- [ ] Add migration guide for existing code
## Testing and Validation
1. **Unit Testing**
- [ ] Create tests for SentenceQueue core functionality
- [ ] Test text processing integration
- [ ] Test audio processing integration
- [ ] Test playback coordination
2. **Integration Testing**
- [ ] Test interaction between SentenceQueue and other modules
- [ ] Validate timing and synchronization
- [ ] Test error handling and recovery
- [ ] Verify performance under load
3. **User Experience Testing**
- [ ] Validate text animation quality
- [ ] Test audio playback quality
- [ ] Verify synchronization from user perspective
- [ ] Test accessibility features
## Implementation Strategy
1. **Phased Rollout**
- [ ] Implement SentenceQueue core structure
- [ ] Add text processing integration
- [ ] Add audio processing integration
- [ ] Implement playback coordination
- [ ] Gradually replace existing components
2. **Backward Compatibility**
- [ ] Maintain support for existing interfaces during transition
- [ ] Implement adapter patterns where needed
- [ ] Add feature flags for enabling/disabling new architecture
- [ ] Create fallback mechanisms for error recovery
3. **Performance Optimization**
- [ ] Implement parallel processing where possible
- [ ] Optimize memory usage for sentence objects
- [ ] Add resource management for audio data
- [ ] Implement efficient queue processing algorithms