Add browser-local Piper WebAssembly speech
Build and publish frontend / publish (push) Failing after 6s
Build and publish frontend / publish (push) Failing after 6s
This commit is contained in:
@@ -12,15 +12,17 @@ The separate `/admin` route provides an authenticated operations view for player
|
||||
|
||||
The game is text- and dialogue-based and uses no images. Place descriptions include sound, smell, air/temperature, touch/surface, and spatial-orientation cues. Visible details may also be described in words, but are never required to understand the scene or play. The transcript is screen-reader announced, and Enter submits the input while Shift+Enter inserts a newline; no separate send button is required.
|
||||
|
||||
Voice output uses the browser's Swedish speech synthesis for all player-facing game dialogue, onboarding, account responses, character and world creation, gameplay replies, in-game errors, and microphone disclosures. The `/admin` route is excluded. The first unauthenticated visit automatically reads an introduction that explains the game, how to create an account, and how to play; the introduction stays visible as text and can be replayed with `Läs introduktionen igen`. New game replies are read automatically and the transcript remains available to screen readers and as text. Returning players hear the latest saved game reply when they reopen the game. Voice settings can change the Swedish voice and speaking rate, but cannot disable automatic speech. These preferences are stored in browser `localStorage`. If the browser lacks speech synthesis or blocks playback, the same content remains available as text and the introduction can be started again with its replay button.
|
||||
Player-facing speech uses Piper TTS in WebAssembly in the browser after audio is activated by a user gesture. It reads game dialogue, onboarding, account responses, character and world creation, gameplay replies, in-game errors, and microphone disclosures; the `/admin` route is excluded. The audio is decoded and played through the Web Audio API, independently of `speechSynthesis`. The first unauthenticated visit keeps the introduction visible and it can be started with `Läs introduktionen`. New game replies are read automatically and the transcript remains available to screen readers and as text. Returning players hear the latest saved game reply when they reopen the game. The player can change speaking rate; narrator pitch is neutral and character pitch is varied consistently by voice identity. These preferences are stored in browser `localStorage`. If WebAssembly, model loading, or audio playback fails, the same content remains available as text and the dialogue offers a retry.
|
||||
|
||||
The microphone button beside the text field uses Web Speech API when available and sends only its final Swedish transcript through the same submission path as typed text. Before its first use, the client explains that the browser may process audio through a provider-selected service; the game itself does not receive or store recordings, while the final transcript is saved as ordinary dialogue. Writing `röstinställningar` or saying `röstkommando alternativ` opens temporary voice controls in the dialogue. The player can select an available Swedish voice and speaking rate and enable or disable short audio cues. Voice preferences are stored in browser `localStorage`.
|
||||
The Piper engine and WebAssembly runtime are served by the frontend from a versioned same-origin path. The Swedish voice model (about 63 MB) downloads on first use from Hugging Face and is cached in browser storage when supported. Only the model files are fetched from that service; dialogue text is synthesized on the device and is not sent to a TTS provider. The frontend CI builds and packages the runtime; there is no Piper server or Kubernetes TTS service.
|
||||
|
||||
When a character or NPC is created, its saved voice identity is masculine, feminine, or neutral. The biography prompt only assigns masculine/feminine when the character is explicitly described that way; otherwise it uses neutral. Speech synthesis applies a stable, subtly varied pitch/rate profile per character, so a character keeps the same sound across visits. Narration always uses neutral pitch. The browser's selected Swedish voice still determines the underlying voice, so the exact sound depends on installed system voices.
|
||||
The microphone button beside the text field uses Web Speech API when available and sends only its final Swedish transcript through the same submission path as typed text. Before its first use, the client explains that the browser may process audio through a provider-selected service; the game itself does not receive or store recordings, while the final transcript is saved as ordinary dialogue. Writing `röstinställningar` or saying `röstkommando alternativ` opens temporary voice controls in the dialogue. The player can adjust speaking rate and enable or disable short audio cues. Voice preferences are stored in browser `localStorage`.
|
||||
|
||||
When a character or NPC is created, its saved voice identity is masculine, feminine, or neutral. The biography prompt only assigns masculine/feminine when the character is explicitly described that way; otherwise it uses neutral. Piper applies a stable, subtle pitch/rate profile per character, so a character keeps the same sound across visits. Narration always uses neutral pitch.
|
||||
|
||||
System voice commands require the prefix `röstkommando`: `upprepa`, `avbryt`, `hjälp`, `alternativ`, or `paus`. Without the prefix, the same words remain ordinary dialogue. Speech recognition and synthesis are browser-dependent; release acceptance must record which real iPhone/Safari/VoiceOver and Android/Chrome/TalkBack combinations were actually checked.
|
||||
|
||||
On iPhone and iPad, Safari may require a direct tap or key press before it permits speech output. The frontend primes speech silently on the first player gesture and then reads game replies automatically. The status reports speech as in progress only after the browser's `onstart` event. Voice settings include **Testa ljudet**; if a game reply does not start, a **Läs senaste repliken** button appears in the dialogue. When Safari reports speech started but the player hears nothing, the status suggests checking device volume and audio output. The introduction is only marked as heard after the browser reports that speech actually started. The speech queue avoids an immediate `cancel()` followed by `speak()`, which older WebKit releases can drop.
|
||||
On iPhone and iPad, Safari requires a user gesture to unlock Web Audio. The frontend resumes the audio context during a trusted tap or key press and then reads generated replies automatically. Voice settings include **Testa ljudet**; if a game reply does not start, a **Läs senaste repliken** button appears in the dialogue. First use can take longer while the voice model downloads; progress and errors are announced in the dialogue. Actual iPhone/iPad playback remains a release acceptance check and must not be claimed until tested on a device.
|
||||
|
||||
The browser and API stay on the same public origin, `https://ai-spel.k3s.k0.nu`. The ingress routes `/` to this frontend service and routes `/api`, `/version`, and `/health` to the Rust service. That keeps the current server-side session cookie same-origin and avoids browser CORS configuration.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user