diff --git a/.gitea/workflows/publish.yaml b/.gitea/workflows/publish.yaml index bc6b62d..53e126a 100644 --- a/.gitea/workflows/publish.yaml +++ b/.gitea/workflows/publish.yaml @@ -39,6 +39,12 @@ jobs: test -s public/tts/piper-wasm-v1/piper/piper_phonemize.data test -s public/tts/piper-wasm-v1/piper-worker.js test -s public/piper-runtime.js + test -s public/tts/piper-wasm-v1/voices/sv/sv_SE/alma/medium/sv_SE-alma-medium.onnx + test -s public/tts/piper-wasm-v1/voices/sv/sv_SE/alma/medium/sv_SE-alma-medium.onnx.json + test -s public/tts/piper-wasm-v1/voices/sv/sv_SE/lisa/medium/sv_SE-lisa-medium.onnx + test -s public/tts/piper-wasm-v1/voices/sv/sv_SE/lisa/medium/sv_SE-lisa-medium.onnx.json + test -s public/tts/piper-wasm-v1/voices/sv/sv_SE/nst/medium/sv_SE-nst-medium.onnx + test -s public/tts/piper-wasm-v1/voices/sv/sv_SE/nst/medium/sv_SE-nst-medium.onnx.json - name: Ensure Helm is available shell: bash diff --git a/AGENTS.md b/AGENTS.md index f688e9e..7849f17 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,5 +7,5 @@ - Changes are written locally and built, checked, published, and exercised by Gitea Actions on the existing `large` runner and Kubernetes. Never run apps, tests, containers, images, or binaries locally. Do not create or register runners. - Never commit secrets, generated credentials, kubeconfigs, or environment files. - Helm charts are published to `harbor.k3s.k0.nu/ai-spel`; Argo CD uses the existing `ai-spel` project and namespace. -- Player-facing speech uses the local Piper WebAssembly adapter when the player has activated audio. The voice model is downloaded once from the configured model repository and cached by the browser; game text is synthesized on-device and must never be sent to a TTS provider. -- Build Piper/ONNX WebAssembly files in the `large` runner and serve them from the frontend image under the versioned `/tts/piper-wasm-v1/` path. Never build or run these assets locally. +- Player-facing speech uses the local Piper WebAssembly adapter when the player has activated audio. All supported Swedish Piper voice models are packaged into the frontend image and served from this origin; model downloads are redirected to same-origin paths and cached by the browser when supported. Game text is synthesized on-device and must never be sent to a TTS provider. +- Build Piper/ONNX WebAssembly files and fetch/checksum pinned Swedish model files in the `large` runner; serve them from the frontend image under `/tts/piper-wasm-v1/`. Never build or run these assets locally. diff --git a/PIPER_VOICE_NOTICES.txt b/PIPER_VOICE_NOTICES.txt new file mode 100644 index 0000000..4a3218a --- /dev/null +++ b/PIPER_VOICE_NOTICES.txt @@ -0,0 +1,25 @@ +Swedish Piper voice models +========================== + +This frontend distributes the Swedish medium Piper models Alma, Lisa, and NST +from the rhasspy/piper-voices repository, pinned to revision +c10ece1aade47bb51c153c893d14e5bf8e5b7117. Model weights are delivered by this +site from /tts/piper-wasm-v1/voices/ after integrity verification in CI. + +Alma (sv_SE-alma-medium) +Source: https://huggingface.co/rhasspy/piper-voices/tree/c10ece1aade47bb51c153c893d14e5bf8e5b7117/sv/sv_SE/alma/medium +The model card states CC BY 4.0 and identifies the upstream voice project at +https://huggingface.co/yeagersthlm/piper-voice-sv-alma. + +Lisa (sv_SE-lisa-medium) +Source: https://huggingface.co/rhasspy/piper-voices/tree/c10ece1aade47bb51c153c893d14e5bf8e5b7117/sv/sv_SE/lisa/medium +The upstream model card describes this as fine-tuned from the Norwegian +talesyntese medium voice. + +NST (sv_SE-nst-medium) +Source: https://huggingface.co/rhasspy/piper-voices/tree/c10ece1aade47bb51c153c893d14e5bf8e5b7117/sv/sv_SE/nst/medium +The upstream model card identifies the NST Swedish TTS dataset as CC0 and +credits KBLab at the National Library of Sweden for training the voice. + +The upstream model files and their model cards remain the source of truth for +license terms. The complete source revision is pinned in scripts/build-piper.mjs. diff --git a/README.md b/README.md index 8df1304..1ff0fea 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ The game is text- and dialogue-based and uses no images. Place descriptions incl Player-facing speech uses Piper TTS in WebAssembly in the browser after audio is activated by a user gesture. It reads game dialogue, onboarding, account responses, character and world creation, gameplay replies, in-game errors, and microphone disclosures; the `/admin` route is excluded. The audio is decoded and played through the Web Audio API, independently of `speechSynthesis`. The first unauthenticated visit keeps the introduction visible and it can be started with `Läs introduktionen`. New game replies are read automatically and the transcript remains available to screen readers and as text. Returning players hear the latest saved game reply when they reopen the game. The player can change speaking rate; narrator pitch is neutral and character pitch is varied consistently by voice identity. These preferences are stored in browser `localStorage`. If WebAssembly, model loading, or audio playback fails, the same content remains available as text and the dialogue offers a retry. -The Piper engine and WebAssembly runtime are served by the frontend from a versioned same-origin path. The Swedish voice model (about 63 MB) downloads on first use from Hugging Face and is cached in browser storage when supported. Only the model files are fetched from that service; dialogue text is synthesized on the device and is not sent to a TTS provider. The frontend CI builds and packages the runtime; there is no Piper server or Kubernetes TTS service. +The Piper engine, WebAssembly runtime, and all three Swedish medium voice models (Alma, Lisa, and NST) are served from this frontend's same-origin `/tts/piper-wasm-v1/` path. The Gitea pipeline downloads the pinned model files, verifies their sizes and cryptographic hashes, and packages them into the versioned frontend image. The browser's Piper adapter redirects the library's built-in model URLs to this same-origin server; it does not request voice files from Hugging Face. A voice model is fetched on first use and cached in browser storage when supported. The game currently uses the NST model. Dialogue text is synthesized on the device and is not sent to a TTS provider. Voice model attribution and license references are available in the voice settings and at `/tts/piper-wasm-v1/voices/ATTRIBUTION.txt`. The microphone button beside the text field uses Web Speech API when available and sends only its final Swedish transcript through the same submission path as typed text. Before its first use, the client explains that the browser may process audio through a provider-selected service; the game itself does not receive or store recordings, while the final transcript is saved as ordinary dialogue. Writing `röstinställningar` or saying `röstkommando alternativ` opens temporary voice controls in the dialogue. The player can adjust speaking rate and enable or disable short audio cues. Voice preferences are stored in browser `localStorage`. diff --git a/index.html b/index.html index 5e8ee1f..ffc8d9f 100644 --- a/index.html +++ b/index.html @@ -259,6 +259,10 @@ voice.value = piperVoice.value; voiceSettings.voiceName = piperVoice.value; voiceLabel.append(voice); fieldset.append(voiceLabel); + const modelCredits = document.createElement('a'); + modelCredits.href = '/tts/piper-wasm-v1/voices/ATTRIBUTION.txt'; + modelCredits.textContent = 'Svenska rösters upphov och licenser'; + fieldset.append(modelCredits); const signalLabel = document.createElement('label'); const signals = document.createElement('input'); signals.type = 'checkbox'; signals.checked = voiceSettings.soundSignals; signalLabel.append(signals, document.createTextNode(' Ljudsignal vid röstinmatning')); fieldset.append(signalLabel); diff --git a/nginx/default.conf b/nginx/default.conf index 12103e7..a2baa0a 100644 --- a/nginx/default.conf +++ b/nginx/default.conf @@ -13,6 +13,8 @@ server { types { application/wasm wasm; application/javascript js; + application/json json; + application/octet-stream onnx; } add_header Cache-Control "public, max-age=31536000, immutable" always; add_header Cross-Origin-Opener-Policy "same-origin" always; diff --git a/piper-model-routing.mjs b/piper-model-routing.mjs new file mode 100644 index 0000000..dabd271 --- /dev/null +++ b/piper-model-routing.mjs @@ -0,0 +1,9 @@ +const PIPER_MODEL_ORIGIN = 'https://huggingface.co/diffusionstudio/piper-voices/resolve/main/'; +const localPathPrefix = '/tts/piper-wasm-v1/voices/'; + +export function localPiperModelUrl(requestedUrl, origin) { + if (!requestedUrl.startsWith(PIPER_MODEL_ORIGIN)) return null; + const relativePath = requestedUrl.slice(PIPER_MODEL_ORIGIN.length); + if (!/^[\w./-]+$/.test(relativePath) || relativePath.split('/').includes('..')) return null; + return new URL(`${localPathPrefix}${relativePath}`, origin).href; +} diff --git a/piper-worker.js b/piper-worker.js index 44f3a3a..5ff91a4 100644 --- a/piper-worker.js +++ b/piper-worker.js @@ -1,7 +1,20 @@ import {TtsSession} from '@mintplex-labs/piper-tts-web'; +import {localPiperModelUrl} from './piper-model-routing.mjs'; const VOICE_ID = 'sv_SE-nst-medium'; const ASSET_ROOT = '/tts/piper-wasm-v1'; +const localFetch = self.fetch.bind(self); + +// The pinned TTS library uses a fixed Hugging Face URL. Keep its OPFS cache logic, +// but serve every model/config request from this frontend origin instead. +self.fetch = (resource, options) => { + const requestedUrl = resource instanceof Request ? resource.url : String(resource); + const localModelUrl = localPiperModelUrl(requestedUrl, self.location.origin); + if (!localModelUrl) return localFetch(resource, options); + const redirected = resource instanceof Request ? new Request(localModelUrl, resource) : localModelUrl; + return localFetch(redirected, options); +}; + let sessionPromise; let inferenceQueue = Promise.resolve(); diff --git a/scripts/build-piper.mjs b/scripts/build-piper.mjs index eff08db..71e15a3 100644 --- a/scripts/build-piper.mjs +++ b/scripts/build-piper.mjs @@ -1,6 +1,10 @@ import {build} from 'esbuild'; -import {cp, mkdir, readdir, rm, stat} from 'node:fs/promises'; +import {createHash} from 'node:crypto'; +import {createWriteStream} from 'node:fs'; +import {cp, mkdir, readdir, rm, stat, rename, unlink} from 'node:fs/promises'; import {join, resolve} from 'node:path'; +import {Readable, Transform} from 'node:stream'; +import {pipeline} from 'node:stream/promises'; const root = process.cwd(); const publicRoot = join(root, 'public'); @@ -10,9 +14,55 @@ const piperSource = join(root, 'node_modules', '@diffusionstudio', 'piper-wasm', const piperWasm = join(piperSource, 'piper_phonemize.wasm'); const piperData = join(piperSource, 'piper_phonemize.data'); +const voiceRevision = 'c10ece1aade47bb51c153c893d14e5bf8e5b7117'; +const voiceRepository = 'https://huggingface.co/rhasspy/piper-voices/resolve'; +const voiceFiles = [ + {path: 'sv/sv_SE/alma/medium/sv_SE-alma-medium.onnx', size: 63434611, sha256: 'f75455eb6afeb1db9e2905d5bd7e4115afcd3710e0d399b6a40981766d32fedc'}, + {path: 'sv/sv_SE/alma/medium/sv_SE-alma-medium.onnx.json', size: 6059, gitBlobSha1: '7eaf3ef20af4a74e9cee9d29b98de3d9fcec0bdc'}, + {path: 'sv/sv_SE/lisa/medium/sv_SE-lisa-medium.onnx', size: 63511038, sha256: '94cae912b31d6e9140d3f5160f1815951588600c7a9e43d539ba1e81a110d131'}, + {path: 'sv/sv_SE/lisa/medium/sv_SE-lisa-medium.onnx.json', size: 7239, gitBlobSha1: 'b2b5fb0f8af2aae4ddfb81eaa7709c6b8faa412b'}, + {path: 'sv/sv_SE/nst/medium/sv_SE-nst-medium.onnx', size: 63104526, sha256: 'df011f56825a59dd1efc080c38a65a1ef70407e60f63050e9246f43a3d7e471e'}, + {path: 'sv/sv_SE/nst/medium/sv_SE-nst-medium.onnx.json', size: 4157, gitBlobSha1: '9766b9769436046a610a505fda925032e12a66b3'}, +]; + +async function downloadVoiceFile(file) { + const destination = join(assetRoot, 'voices', file.path); + const temporary = `${destination}.part`; + await mkdir(join(destination, '..'), {recursive: true}); + const response = await fetch(`${voiceRepository}/${voiceRevision}/${file.path}`); + if (!response.ok || !response.body) { + throw new Error(`Unable to download Piper voice asset ${file.path}: HTTP ${response.status}`); + } + const sha256 = createHash('sha256'); + const sha1 = createHash('sha1').update(Buffer.from(`blob ${file.size}\0`)); + let size = 0; + const verifyAndPassThrough = new Transform({ + transform(chunk, _encoding, callback) { + size += chunk.length; + sha256.update(chunk); + sha1.update(chunk); + callback(null, chunk); + }, + }); + try { + await pipeline(Readable.fromWeb(response.body), verifyAndPassThrough, createWriteStream(temporary)); + const actualSha1 = sha1.digest('hex'); + if (size !== file.size || (file.sha256 && sha256.digest('hex') !== file.sha256) || + (file.gitBlobSha1 && actualSha1 !== file.gitBlobSha1)) { + throw new Error(`Piper voice asset integrity check failed for ${file.path} (${size} bytes)`); + } + await rename(temporary, destination); + } catch (error) { + await unlink(temporary).catch(() => {}); + throw error; + } +} + await rm(assetRoot, {recursive: true, force: true}); await mkdir(join(assetRoot, 'onnx'), {recursive: true}); await mkdir(join(assetRoot, 'piper'), {recursive: true}); +for (const file of voiceFiles) await downloadVoiceFile(file); +await cp(join(root, 'PIPER_VOICE_NOTICES.txt'), join(assetRoot, 'voices', 'ATTRIBUTION.txt')); const wasmFiles = (await readdir(onnxSource)) .filter(name => name.endsWith('.wasm')); @@ -50,4 +100,14 @@ await build({ legalComments: 'none', }); +for (const file of voiceFiles) { + const downloaded = join(assetRoot, 'voices', file.path); + if ((await stat(downloaded)).size !== file.size) throw new Error(`Missing Piper voice asset: ${downloaded}`); +} +const workerBundle = await (await import('node:fs/promises')).readFile(join(assetRoot, 'piper-worker.js'), 'utf8'); +if (!workerBundle.includes('https://huggingface.co/diffusionstudio/piper-voices/resolve/main/') || + !workerBundle.includes('/tts/piper-wasm-v1/voices/')) { + throw new Error('The Piper worker is missing its same-origin model download route.'); +} + console.log(`Piper browser assets prepared at ${assetRoot}`); diff --git a/scripts/check-frontend.sh b/scripts/check-frontend.sh index 6a704dc..bbff805 100755 --- a/scripts/check-frontend.sh +++ b/scripts/check-frontend.sh @@ -46,6 +46,11 @@ grep -q 'sv_SE-nst-medium' piper-worker.js grep -q 'new Worker(workerUrl)' piper-runtime.js grep -q 'worker.js?v=' piper-runtime.js grep -q 'worker.onmessageerror' piper-runtime.js +grep -q 'localPiperModelUrl' piper-worker.js +grep -q 'PIPER_MODEL_ORIGIN' piper-model-routing.mjs +grep -q '/tts/piper-wasm-v1/voices/' piper-model-routing.mjs +grep -q 'same-origin.*voice' README.md +grep -q 'ATTRIBUTION.txt' index.html grep -q "SpeechRecognition || win.webkitSpeechRecognition" voice.js grep -q "submitMessage(text, 'voice')" index.html grep -q 'Spelets repliker läses alltid upp' index.html @@ -80,6 +85,6 @@ if sed -n '/
/p' index.html | grep ' { + const {localPiperModelUrl} = await import('../piper-model-routing.mjs'); + const origin = 'https://ai-spel.k3s.k0.nu'; + assert.equal( + localPiperModelUrl('https://huggingface.co/diffusionstudio/piper-voices/resolve/main/sv/sv_SE/nst/medium/sv_SE-nst-medium.onnx', origin), + 'https://ai-spel.k3s.k0.nu/tts/piper-wasm-v1/voices/sv/sv_SE/nst/medium/sv_SE-nst-medium.onnx', + ); + assert.equal( + localPiperModelUrl('https://huggingface.co/diffusionstudio/piper-voices/resolve/main/sv/sv_SE/lisa/medium/sv_SE-lisa-medium.onnx.json', origin), + 'https://ai-spel.k3s.k0.nu/tts/piper-wasm-v1/voices/sv/sv_SE/lisa/medium/sv_SE-lisa-medium.onnx.json', + ); +}); + +test('Piper model routing leaves unrelated URLs alone and rejects traversal paths', async () => { + const {localPiperModelUrl} = await import('../piper-model-routing.mjs'); + const origin = 'https://ai-spel.k3s.k0.nu'; + assert.equal(localPiperModelUrl('https://example.com/model.onnx', origin), null); + assert.equal(localPiperModelUrl('https://huggingface.co.evil.test/diffusionstudio/piper-voices/resolve/main/model.onnx', origin), null); + assert.equal(localPiperModelUrl('https://huggingface.co/diffusionstudio/piper-voices/resolve/main/../secret', origin), null); +});