[ { "path": ".gitignore", "content": ".DS_Store\n.idea\n*.log\ntmp/\n\n*.py[cod]\n*.egg\n/build\nhtmlcov\n\n.projectile\n.venv/\nvenv/\n.mypy_cache/\n*.egg-info/\n\n/local/\n" }, { "path": ".gitmodules", "content": "[submodule \"programs/asr/whisper.cpp/build/whisper.cpp\"]\n\tpath = programs/asr/whisper.cpp/build/whisper.cpp\n\turl = https://github.com/ggerganov/whisper.cpp\n" }, { "path": ".isort.cfg", "content": "[settings]\nmulti_line_output=3\ninclude_trailing_comma=True\nforce_grid_wrap=0\nuse_parentheses=True\nline_length=88\n" }, { "path": "LICENSE.md", "content": "MIT License\n\nCopyright (c) 2022 Michael Hansen\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "![Rhasspy 3](img/banner.png)\n\n**NOTE: This is a very early developer preview!**\n\nAn open source toolkit for building voice assistants.\n\n![Voice assistant pipeline](img/pipeline.png)\n\nRhasspy focuses on:\n\n* Privacy - no data leaves your computer unless you want it to\n* Broad language support - more than just English\n* Customization - everything can be changed\n\n## Getting Started\n\n* Check out the [tutorial](docs/tutorial.md)\n* Connect Rhasspy to [Home Assistant](docs/home_assistant.md)\n * Install the [Rhasspy 3 add-on](https://github.com/rhasspy/hassio-addons)\n* Run one or more [satellites](docs/satellite.md)\n* Join [the community](https://community.rhasspy.org/)\n\n\n## Missing Pieces\n\nThis is a developer preview, so there are lots of things missing:\n\n* A user friendly web UI\n* An automated method for installing programs/services and downloading models\n* Support for custom speech to text grammars\n* Intent systems besides Home Assistant\n* The ability to accumulate context within a pipeline\n\n\n## Core Concepts\n\n### Domains\n\nRhasspy is organized by [domain](docs/domains.md):\n\n* mic - audio input\n* wake - wake word detection\n* asr - speech to text\n* vad - voice activity detection\n* intent - intent recognition from text\n* handle - intent or text input handling\n* tts - text to speech\n* snd - audio output\n\n\n### Programs\n\nRhasspy talks to external programs using the [Wyoming protocol](docs/wyoming.md). You can add your own programs by implementing the protocol or using an [adapter](#adapters).\n\n\n### Adapters\n\n[Small scripts](docs/adapters.md) that live in `bin/` and bridge existing programs into the [Wyoming protocol](docs/wyoming.md).\n\nFor example, a speech to text program (`asr`) that accepts a WAV file and outputs text can use `asr_adapter_wav2text.py`\n\n\n### Pipelines\n\nComplete voice loop from microphone input (mic) to speaker output (snd). Stages are:\n\n1. detect (optional)\n * Wait until wake word is detected in mic\n2. transcribe\n * Listen until vad detects silence, then convert audio to text\n3. recognize (optional)\n * Recognize an intent from text\n4. handle\n * Handle an intent or text, producing a text response\n5. speak\n * Convert handle output text to speech, and speak through snd\n\n### Servers\n\nSome programs take a while to load, so it's best to leave them running as a server. Use `bin/server_run.py` or add `--server ` when running the HTTP server.\n\nSee `servers` section of `configuration.yaml` file.\n\n---\n\n\n## Supported Programs\n\n* mic\n * [arecord](https://alsa-project.org/wiki/Main_Page)\n * [gstreamer_udp](https://gstreamer.freedesktop.org/)\n * [sounddevice](https://python-sounddevice.readthedocs.io)\n * [pyaudio](https://people.csail.mit.edu/hubert/pyaudio/docs/)\n* wake \n * [porcupine1](https://github.com/Picovoice/porcupine)\n * [precise-lite](https://github.com/mycroftAI/mycroft-precise)\n * [snowboy](https://github.com/Kitt-AI/snowboy)\n* vad\n * [silero](https://github.com/snakers4/silero-vad)\n * [webrtcvad](https://pypi.org/project/webrtcvad/)\n* asr \n * [whisper](https://github.com/openai/whisper)\n * [whisper-cpp](https://github.com/ggerganov/whisper.cpp/)\n * [faster-whisper](https://github.com/guillaumekln/faster-whisper/)\n * [vosk](https://alphacephei.com/vosk/)\n * [coqui-stt](https://stt.readthedocs.io)\n * [pocketsphinx](https://github.com/cmusphinx/pocketsphinx)\n* handle\n * [home_assistant_conversation](https://www.home-assistant.io/docs/assist)\n* tts \n * [piper](https://github.com/rhasspy/piper/)\n * [mimic3](https://github.com/mycroftAI/mimic3)\n * [larynx](https://github.com/rhasspy/larynx/)\n * [coqui-tts](https://tts.readthedocs.io)\n * [marytts](http://mary.dfki.de/)\n * [flite](http://www.festvox.org/flite/)\n * [festival](http://www.cstr.ed.ac.uk/projects/festival/)\n * [espeak-ng](https://github.com/espeak-ng/espeak-ng/)\n* snd\n * [aplay](https://alsa-project.org/wiki/Main_Page)\n * [gstreamer_udp](https://gstreamer.freedesktop.org/)\n \n \n---\n\n\n## HTTP API\n\n`http://localhost:13331/`\n\nUnless overridden, the pipeline named \"default\" is used.\n\n* `/pipeline/run`\n * Runs a full pipeline from mic to snd\n * Produces JSON\n * Override `pipeline` or:\n * `wake_program`\n * `asr_program`\n * `intent_program`\n * `handle_program`\n * `tts_program`\n * `snd_program`\n * Skip stages with `start_after`\n * `wake` - skip detection, body is detection name (text)\n * `asr` - skip recording, body is transcript (text) or WAV audio\n * `intent` - skip recognition, body is intent/not-recognized event (JSON)\n * `handle` - skip handling, body is handle/not-handled event (JSON)\n * `tts` - skip synthesis, body is WAV audio\n * Stop early with `stop_after`\n * `wake` - only detection\n * `asr` - detection and transcription\n * `intent` - detection, transcription, recognition\n * `handle` - detection, transcription, recognition, handling\n * `tts` - detection, transcription, recognition, handling, synthesis\n* `/wake/detect`\n * Detect wake word in WAV input\n * Produces JSON\n * Override `wake_program` or `pipeline`\n* `/asr/transcribe`\n * Transcribe audio from WAV input\n * Produces JSON\n * Override `asr_program` or `pipeline`\n* `/intent/recognize`\n * Recognizes intent from text body (POST) or `text` (GET)\n * Produces JSON\n * Override `intent_program` or `pipeline`\n* `/handle/handle`\n * Handles intent/text from body (POST) or `input` (GET)\n * `Content-Type` must be `application/json` for intent input\n * Override `handle_program` or `pipeline`\n* `/tts/synthesize`\n * Synthesizes audio from text body (POST) or `text` (GET)\n * Produces WAV audio\n * Override `tts_program` or `pipeline`\n* `/tts/speak`\n * Plays audio from text body (POST) or `text` (GET)\n * Produces JSON\n * Override `tts_program`, `snd_program`, or `pipeline`\n* `/snd/play`\n * Plays WAV audio via snd\n * Override `snd_program` or `pipeline`\n* `/config`\n * Returns JSON config\n* `/version`\n * Returns version info\n\n\n## WebSocket API\n\n`ws://localhost:13331/`\n\nAudio streams are raw PCM in binary messages.\n\nUse the `rate`, `width`, and `channels` parameters for sample rate (hertz), width (bytes), and channel count. By default, input audio is 16Khz 16-bit mono, and output audio is 22Khz 16-bit mono.\n\nThe client can \"end\" the audio stream by sending an empty binary message.\n\n* `/pipeline/asr-tts`\n * Run pipeline from asr (stream in) to tts (stream out)\n * Produces JSON messages as events happen\n * Override `pipeline` or:\n * `asr_program`\n * `vad_program`\n * `handle_program`\n * `tts_program`\n * Use `in_rate`, `in_width`, `in_channels` for audio input format\n * Use `out_rate`, `out_width`, `out_channels` for audio output format\n* `/wake/detect`\n * Detect wake word from websocket audio stream\n * Produces a JSON message when audio stream ends\n * Override `wake_program` or `pipeline`\n* `/asr/transcribe`\n * Transcribe a websocket audio stream\n * Produces a JSON message when audio stream ends\n * Override `asr_program` or `pipeline`\n* `/snd/play`\n * Play a websocket audio stream\n * Produces a JSON message when audio stream ends\n * Override `snd_program` or `pipeline`\n" }, { "path": "bin/asr_adapter_raw2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport shlex\nimport subprocess\nfrom pathlib import Path\n\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.audio import AudioChunk, AudioChunkConverter, AudioStop\nfrom rhasspy3.event import read_event, write_event\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\"--shell\", action=\"store_true\")\n #\n parser.add_argument(\n \"--rate\",\n type=int,\n help=\"Sample rate (hz)\",\n )\n parser.add_argument(\n \"--width\",\n type=int,\n help=\"Sample width bytes\",\n )\n parser.add_argument(\n \"--channels\",\n type=int,\n help=\"Sample channel count\",\n )\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n if args.shell:\n command = args.command\n else:\n command = shlex.split(args.command)\n\n proc = subprocess.Popen(\n command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=args.shell\n )\n text = \"\"\n converter = AudioChunkConverter(args.rate, args.width, args.channels)\n\n with proc:\n assert proc.stdin is not None\n assert proc.stdout is not None\n\n while True:\n event = read_event()\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n chunk = converter.convert(chunk)\n proc.stdin.write(chunk.audio)\n proc.stdin.flush()\n elif AudioStop.is_type(event.type):\n break\n\n stdout, _stderr = proc.communicate()\n text = stdout.decode()\n\n write_event(Transcript(text=text.strip()).event())\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/asr_adapter_wav2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport shlex\nimport subprocess\nimport tempfile\nimport wave\nfrom pathlib import Path\n\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.audio import AudioChunk, AudioStop\nfrom rhasspy3.event import read_event, write_event\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\"--shell\", action=\"store_true\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n with tempfile.NamedTemporaryFile(mode=\"wb+\", suffix=\".wav\") as wav_io:\n args.command = args.command.format(wav_file=wav_io.name)\n if args.shell:\n command = args.command\n else:\n command = shlex.split(args.command)\n\n wav_params_set = False\n wav_file: wave.Wave_write = wave.open(wav_io, \"wb\")\n try:\n with wav_file:\n while True:\n event = read_event()\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n if not wav_params_set:\n wav_file.setframerate(chunk.rate)\n wav_file.setsampwidth(chunk.width)\n wav_file.setnchannels(chunk.channels)\n wav_params_set = True\n\n wav_file.writeframes(chunk.audio)\n elif AudioStop.is_type(event.type):\n break\n\n wav_io.seek(0)\n text = subprocess.check_output(command, shell=args.shell).decode()\n write_event(Transcript(text=text.strip()).event())\n except wave.Error:\n pass\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/asr_transcribe.py", "content": "#!/usr/bin/env python3\n\"\"\"Transcribes mic audio into text.\"\"\"\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport sys\nfrom pathlib import Path\n\nfrom rhasspy3.asr import DOMAIN, Transcript\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import async_read_event\nfrom rhasspy3.mic import DOMAIN as MIC_DOMAIN\nfrom rhasspy3.program import create_process\nfrom rhasspy3.vad import segment\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--mic-program\", help=\"Name of mic program to use (overrides pipeline)\"\n )\n parser.add_argument(\n \"--asr-program\", help=\"Name of asr program to use (overrides pipeline)\"\n )\n parser.add_argument(\n \"--vad-program\", help=\"Name of vad program to use (overrides pipeline)\"\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n mic_program = args.mic_program\n asr_program = args.asr_program\n vad_program = args.vad_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not mic_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n mic_program = pipeline.mic\n\n assert mic_program, \"No mic program\"\n\n if not asr_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n asr_program = pipeline.asr\n\n assert asr_program, \"No asr program\"\n\n if not vad_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n vad_program = pipeline.vad\n\n assert vad_program, \"No vad program\"\n\n # Transcribe voice command\n async with (await create_process(rhasspy, MIC_DOMAIN, mic_program)) as mic_proc, (\n await create_process(rhasspy, DOMAIN, asr_program)\n ) as asr_proc:\n assert mic_proc.stdout is not None\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n\n _LOGGER.info(\"Ready\")\n await segment(rhasspy, vad_program, mic_proc.stdout, asr_proc.stdin)\n\n # Read transcript\n _LOGGER.debug(\"Waiting for transcript\")\n transcript = Transcript(text=\"\")\n while True:\n event = await async_read_event(asr_proc.stdout)\n if event is None:\n break\n\n if Transcript.is_type(event.type):\n transcript = Transcript.from_event(event)\n break\n\n json.dump(transcript.event().to_dict(), sys.stdout, ensure_ascii=False)\n print(\"\", flush=True)\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/asr_transcribe_stream.py", "content": "#!/usr/bin/env python3\n\"\"\"Transcribes raw audio from stdin into text.\"\"\"\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport sys\nfrom pathlib import Path\n\nfrom rhasspy3.asr import DOMAIN, Transcript\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n DEFAULT_SAMPLES_PER_CHUNK,\n AudioChunk,\n AudioChunkConverter,\n AudioStart,\n AudioStop,\n)\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import async_read_event, async_write_event\nfrom rhasspy3.program import create_process\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--asr-program\", help=\"Name of asr program to use (overrides pipeline)\"\n )\n #\n parser.add_argument(\n \"--mic-rate\",\n type=int,\n default=DEFAULT_IN_RATE,\n help=\"Input sample rate (hertz)\",\n )\n parser.add_argument(\n \"--mic-width\",\n type=int,\n default=DEFAULT_IN_WIDTH,\n help=\"Input sample width (bytes)\",\n )\n parser.add_argument(\n \"--mic-channels\",\n type=int,\n default=DEFAULT_IN_CHANNELS,\n help=\"Input sample channel count\",\n )\n #\n parser.add_argument(\n \"--asr-rate\", type=int, default=DEFAULT_IN_RATE, help=\"asr sample rate (hertz)\"\n )\n parser.add_argument(\n \"--asr-width\",\n type=int,\n default=DEFAULT_IN_WIDTH,\n help=\"asr sample width (bytes)\",\n )\n parser.add_argument(\n \"--asr-channels\",\n type=int,\n default=DEFAULT_IN_CHANNELS,\n help=\"asr sample channel count\",\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=DEFAULT_SAMPLES_PER_CHUNK,\n help=\"Samples to process per chunk\",\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n asr_program = args.asr_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not asr_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n asr_program = pipeline.asr\n\n assert asr_program, \"No asr program\"\n _LOGGER.debug(\"asr program: %s\", asr_program)\n\n # Transcribe raw audio from stdin\n converter = AudioChunkConverter(args.asr_rate, args.asr_width, args.asr_channels)\n bytes_per_chunk = args.samples_per_chunk * args.mic_width * args.mic_channels\n timestamp = 0\n\n async with (await create_process(rhasspy, DOMAIN, asr_program)) as asr_proc:\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n _LOGGER.debug(\"Started %s\", asr_program)\n\n await async_write_event(\n AudioStart(\n args.asr_rate, args.asr_width, args.asr_channels, timestamp=timestamp\n ).event(),\n asr_proc.stdin,\n )\n\n audio_bytes = sys.stdin.buffer.read(bytes_per_chunk)\n while audio_bytes:\n chunk = AudioChunk(\n args.mic_rate,\n args.mic_width,\n args.mic_channels,\n audio_bytes,\n timestamp=timestamp,\n )\n timestamp += chunk.milliseconds\n chunk = converter.convert(chunk)\n\n # Write audio\n await async_write_event(\n chunk.event(),\n asr_proc.stdin,\n )\n audio_bytes = sys.stdin.buffer.read(bytes_per_chunk)\n\n await async_write_event(AudioStop(timestamp=timestamp).event(), asr_proc.stdin)\n\n # Read transcript\n transcript = Transcript(text=\"\")\n while True:\n event = await async_read_event(asr_proc.stdout)\n if event is None:\n break\n\n if Transcript.is_type(event.type):\n transcript = Transcript.from_event(event)\n break\n\n json.dump(transcript.event().to_dict(), sys.stdout, ensure_ascii=False)\n print(\"\", flush=True)\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/asr_transcribe_wav.py", "content": "#!/usr/bin/env python3\n\"\"\"Transcribes WAV audio into text.\"\"\"\nimport argparse\nimport asyncio\nimport io\nimport json\nimport logging\nimport os\nimport sys\nimport time\nimport wave\nfrom pathlib import Path\nfrom typing import Iterable, Optional\n\nfrom rhasspy3.asr import DOMAIN, Transcript\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n DEFAULT_SAMPLES_PER_CHUNK,\n AudioChunkConverter,\n AudioStart,\n AudioStop,\n wav_to_chunks,\n)\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import async_read_event, async_write_event\nfrom rhasspy3.program import create_process\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--asr-program\", help=\"Name of asr program to use (overrides pipeline)\"\n )\n #\n parser.add_argument(\n \"--rate\", type=int, default=DEFAULT_IN_RATE, help=\"Sample rate (hertz)\"\n )\n parser.add_argument(\n \"--width\", type=int, default=DEFAULT_IN_WIDTH, help=\"Sample width (bytes)\"\n )\n parser.add_argument(\n \"--channels\", type=int, default=DEFAULT_IN_CHANNELS, help=\"Sample channel count\"\n )\n parser.add_argument(\n \"--samples-per-chunk\", type=int, default=DEFAULT_SAMPLES_PER_CHUNK\n )\n #\n parser.add_argument(\"wav\", nargs=\"*\", help=\"Path to WAV file(s)\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n asr_program = args.asr_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not asr_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n asr_program = pipeline.asr\n\n assert asr_program, \"No asr program\"\n _LOGGER.debug(\"asr program: %s\", asr_program)\n\n # Transcribe WAV file(s)\n for wav_bytes in get_wav_bytes(args):\n converter = AudioChunkConverter(args.rate, args.width, args.channels)\n\n with io.BytesIO(wav_bytes) as wav_io:\n with wave.open(wav_io, \"rb\") as wav_file:\n chunks = list(wav_to_chunks(wav_file, args.samples_per_chunk))\n\n async with (await create_process(rhasspy, DOMAIN, asr_program)) as asr_proc:\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n\n # Write audio\n start_time = time.monotonic_ns()\n await async_write_event(\n AudioStart(args.rate, args.width, args.channels, timestamp=0).event(),\n asr_proc.stdin,\n )\n\n last_timestamp: Optional[int] = None\n for chunk in chunks:\n chunk = converter.convert(chunk)\n await async_write_event(chunk.event(), asr_proc.stdin)\n last_timestamp = chunk.timestamp\n\n await async_write_event(\n AudioStop(timestamp=last_timestamp).event(),\n asr_proc.stdin,\n )\n\n # Read transcript\n _LOGGER.debug(\"Waiting for transcription\")\n transcript = Transcript(text=\"\")\n while True:\n event = await async_read_event(asr_proc.stdout)\n if event is None:\n break\n\n if Transcript.is_type(event.type):\n transcript = Transcript.from_event(event)\n end_time = time.monotonic_ns()\n _LOGGER.debug(\n \"Transcribed in %s second(s)\", (end_time - start_time) / 1e9\n )\n break\n\n _LOGGER.debug(transcript)\n\n json.dump(transcript.event().to_dict(), sys.stdout, ensure_ascii=False)\n print(\"\", flush=True)\n\n\ndef get_wav_bytes(args: argparse.Namespace) -> Iterable[bytes]:\n \"\"\"Yields WAV audio from stdin or args.\"\"\"\n if args.wav:\n # WAV file path(s)\n for wav_path in args.wav:\n _LOGGER.debug(\"Processing %s\", wav_path)\n with open(wav_path, \"rb\") as wav_file:\n yield wav_file.read()\n else:\n # WAV on stdin\n if os.isatty(sys.stdin.fileno()):\n print(\"Reading WAV audio from stdin\", file=sys.stderr)\n\n yield sys.stdin.buffer.read()\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/client_unix_socket.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport socket\nimport threading\n\nfrom rhasspy3.event import read_event, write_event\n\n_LOGGER = logging.getLogger(\"wrapper_unix_socket\")\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\"socketfile\", help=\"Path to Unix domain socket file\")\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n _LOGGER.debug(\"Connecting to %s\", args.socketfile)\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.connect(args.socketfile)\n _LOGGER.debug(\"Connected\")\n\n try:\n with sock.makefile(mode=\"rwb\") as conn_file:\n read_thread = threading.Thread(\n target=read_proc, args=(conn_file,), daemon=True\n )\n read_thread.start()\n\n write_thread = threading.Thread(\n target=write_proc, args=(conn_file,), daemon=True\n )\n write_thread.start()\n write_thread.join()\n except KeyboardInterrupt:\n pass\n\n\ndef read_proc(conn_file):\n try:\n while True:\n event = read_event(conn_file)\n if event is None:\n break\n\n write_event(event)\n except Exception:\n _LOGGER.exception(\"Unexpected error in read thread\")\n\n\ndef write_proc(conn_file):\n try:\n while True:\n event = read_event()\n if event is None:\n break\n\n write_event(event, conn_file)\n except Exception:\n _LOGGER.exception(\"Unexpected error in write thread\")\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/config_print.py", "content": "#!/usr/bin/env python3\n\"\"\"Prints configuration as JSON.\"\"\"\nimport argparse\nimport json\nimport logging\nimport sys\nfrom pathlib import Path\n\nfrom rhasspy3.core import Rhasspy\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\"--indent\", type=int, default=4)\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n json.dump(rhasspy.config_dict, sys.stdout, indent=args.indent, ensure_ascii=False)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/handle_adapter_json.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport shlex\nimport subprocess\nfrom pathlib import Path\n\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.handle import Handled, NotHandled\nfrom rhasspy3.intent import Intent, NotRecognized\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\"--shell\", action=\"store_true\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n if args.shell:\n command = args.command\n else:\n command = shlex.split(args.command)\n\n proc = subprocess.Popen(\n command,\n stdin=subprocess.PIPE,\n stdout=subprocess.PIPE,\n shell=args.shell,\n universal_newlines=True,\n )\n with proc:\n assert proc.stdin is not None\n assert proc.stdout is not None\n\n while True:\n event = read_event()\n if event is None:\n break\n\n if Intent.is_type(event.type):\n intent = Intent.from_event(event)\n stdout, _stderr = proc.communicate(\n input=json.dumps(\n {\n \"intent\": {\n \"name\": intent.name,\n },\n \"entities\": [\n {\"entity\": entity.name, \"value\": entity.value}\n for entity in intent.entities or []\n ],\n \"slots\": {\n entity.name: entity.value\n for entity in intent.entities or []\n },\n },\n ensure_ascii=False,\n )\n )\n handled = False\n for line in stdout.splitlines():\n line = line.strip()\n if line:\n write_event(Handled(text=line).event())\n handled = True\n break\n\n if not handled:\n write_event(NotHandled().event())\n\n break\n\n if NotRecognized.is_type(event.type):\n write_event(NotHandled().event())\n break\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/handle_adapter_text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport shlex\nimport subprocess\nfrom pathlib import Path\n\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.handle import Handled, NotHandled\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\"--shell\", action=\"store_true\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n if args.shell:\n command = args.command\n else:\n command = shlex.split(args.command)\n\n proc = subprocess.Popen(\n command,\n stdin=subprocess.PIPE,\n stdout=subprocess.PIPE,\n shell=args.shell,\n universal_newlines=True,\n )\n with proc:\n assert proc.stdin is not None\n assert proc.stdout is not None\n\n while True:\n event = read_event()\n if event is None:\n break\n\n if Transcript.is_type(event.type):\n transcript = Transcript.from_event(event)\n stdout, _stderr = proc.communicate(input=transcript.text)\n handled = False\n for line in stdout.splitlines():\n line = line.strip()\n if line:\n write_event(Handled(text=line).event())\n handled = True\n break\n\n if not handled:\n write_event(NotHandled().event())\n\n break\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/handle_intent.py", "content": "#!/usr/bin/env python3\n\"\"\"Handle text or intent.\"\"\"\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport os\nimport sys\nfrom pathlib import Path\nfrom typing import Iterable\n\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.handle import handle\nfrom rhasspy3.intent import Intent\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--handle-program\", help=\"Name of handle program to use (overrides pipeline)\"\n )\n parser.add_argument(\"intent\", nargs=\"*\", help=\"Intent JSON event(s) to handle\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n handle_program = args.handle_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not handle_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n handle_program = pipeline.handle\n\n assert handle_program, \"No handle program\"\n\n for line in get_input(args):\n # Intent JSON\n handle_input: Intent = Intent.from_dict(json.loads(line))\n handle_result = await handle(rhasspy, handle_program, handle_input)\n if handle_result is None:\n _LOGGER.warning(\"No result\")\n continue\n\n _LOGGER.debug(handle_result)\n json.dump(handle_result.event().to_dict(), sys.stdout, ensure_ascii=False)\n\n\ndef get_input(args: argparse.Namespace) -> Iterable[str]:\n \"\"\"Get input from stdin or args.\"\"\"\n if args.intent:\n for event_json in args.intent:\n yield event_json\n else:\n if os.isatty(sys.stdin.fileno()):\n print(\"Reading input from stdin\", file=sys.stderr)\n\n for line in sys.stdin:\n line = line.strip()\n if line:\n yield line\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/handle_text.py", "content": "#!/usr/bin/env python3\n\"\"\"Handle text or intent.\"\"\"\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport os\nimport sys\nfrom pathlib import Path\nfrom typing import Iterable\n\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.handle import handle\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--handle-program\", help=\"Name of handle program to use (overrides pipeline)\"\n )\n parser.add_argument(\"text\", nargs=\"*\", help=\"Text input to handle\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n handle_program = args.handle_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not handle_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n handle_program = pipeline.handle\n\n assert handle_program, \"No handle program\"\n\n for line in get_input(args):\n # Text\n handle_input = Transcript(text=line)\n handle_result = await handle(rhasspy, handle_program, handle_input)\n if handle_result is None:\n _LOGGER.warning(\"No result\")\n continue\n\n _LOGGER.debug(handle_result)\n json.dump(handle_result.event().to_dict(), sys.stdout, ensure_ascii=False)\n\n\ndef get_input(args: argparse.Namespace) -> Iterable[str]:\n \"\"\"Get input from stdin or args.\"\"\"\n if args.text:\n for text in args.text:\n yield text\n else:\n if os.isatty(sys.stdin.fileno()):\n print(\"Reading input from stdin\", file=sys.stderr)\n\n for line in sys.stdin:\n line = line.strip()\n if line:\n yield line\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/intent_recognize.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport os\nimport sys\nfrom pathlib import Path\nfrom typing import Iterable\n\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.intent import recognize\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--intent-program\", help=\"Name of intent program to use (overrides pipeline)\"\n )\n parser.add_argument(\"text\", nargs=\"*\", help=\"Text to recognize\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n intent_program = args.intent_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not intent_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n intent_program = pipeline.intent\n\n assert intent_program, \"No intent program\"\n\n for text in get_texts(args):\n intent_result = await recognize(rhasspy, intent_program, text)\n if intent_result is None:\n continue\n\n json.dump(intent_result.event().data, sys.stdout, ensure_ascii=False)\n print(\"\", flush=True)\n\n\ndef get_texts(args: argparse.Namespace) -> Iterable[str]:\n if args.text:\n for text in args.text:\n yield text\n else:\n if os.isatty(sys.stdin.fileno()):\n print(\"Reading text from stdin\", file=sys.stderr)\n\n for line in sys.stdin:\n line = line.strip()\n if line:\n yield line\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/mic_adapter_raw.py", "content": "#!/usr/bin/env python3\n\"\"\"Reads raw audio chunks from stdin.\"\"\"\nimport argparse\nimport logging\nimport shlex\nimport subprocess\nimport time\nfrom pathlib import Path\n\nfrom rhasspy3.audio import DEFAULT_SAMPLES_PER_CHUNK, AudioChunk, AudioStart\nfrom rhasspy3.event import write_event\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\"--shell\", action=\"store_true\", help=\"Run command with shell\")\n #\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=DEFAULT_SAMPLES_PER_CHUNK,\n help=\"Number of samples to read at a time from command\",\n )\n parser.add_argument(\n \"--rate\",\n type=int,\n required=True,\n help=\"Sample rate (hz)\",\n )\n parser.add_argument(\n \"--width\",\n type=int,\n required=True,\n help=\"Sample width bytes\",\n )\n parser.add_argument(\n \"--channels\",\n type=int,\n required=True,\n help=\"Sample channel count\",\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n bytes_per_chunk = args.samples_per_chunk * args.width * args.channels\n\n if args.shell:\n command = args.command\n else:\n command = shlex.split(args.command)\n\n proc = subprocess.Popen(command, stdout=subprocess.PIPE)\n with proc:\n assert proc.stdout is not None\n\n write_event(\n AudioStart(\n args.rate, args.width, args.channels, timestamp=time.monotonic_ns()\n ).event()\n )\n while True:\n audio_bytes = proc.stdout.read(bytes_per_chunk)\n if not audio_bytes:\n break\n\n write_event(\n AudioChunk(\n args.rate,\n args.width,\n args.channels,\n audio_bytes,\n timestamp=time.monotonic_ns(),\n ).event()\n )\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/mic_record_sample.py", "content": "#!/usr/bin/env python3\n\"\"\"Record a spoken audio sample to a WAV file.\"\"\"\nimport argparse\nimport asyncio\nimport logging\nimport wave\nfrom collections import deque\nfrom pathlib import Path\nfrom typing import Deque\n\nfrom rhasspy3.audio import AudioChunk\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import async_read_event, async_write_event\nfrom rhasspy3.mic import DOMAIN as MIC_DOMAIN\nfrom rhasspy3.program import create_process\nfrom rhasspy3.vad import DOMAIN as VAD_DOMAIN\nfrom rhasspy3.vad import VoiceStarted, VoiceStopped\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path to WAV file(s) to write\")\n #\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--mic-program\", help=\"Name of mic program to use (overrides pipeline)\"\n )\n parser.add_argument(\n \"--vad-program\", help=\"Name of vad program to use (overrides pipeline)\"\n )\n #\n parser.add_argument(\n \"--chunk-buffer-size\",\n type=int,\n default=25,\n help=\"Audio chunks to buffer before start is known\",\n )\n parser.add_argument(\n \"-b\",\n \"--keep-chunks-before\",\n type=int,\n default=5,\n help=\"Audio chunks to keep before voice starts\",\n )\n parser.add_argument(\n \"-a\",\n \"--keep-chunks-after\",\n type=int,\n default=0,\n help=\"Audio chunks to keep after voice ends\",\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n mic_program = args.mic_program\n vad_program = args.vad_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not mic_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n mic_program = pipeline.mic\n\n assert mic_program, \"No mic program\"\n\n if not vad_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n vad_program = pipeline.vad\n\n assert vad_program, \"No vad program\"\n\n for wav_path in args.wav_file:\n wav_file: wave.Wave_write = wave.open(wav_path, \"wb\")\n with wav_file:\n is_first_chunk = True\n\n # Audio kept before we get the event that the voice command started\n # at a timestep in the past.\n chunk_buffer: Deque[AudioChunk] = deque(\n maxlen=max(args.chunk_buffer_size, args.keep_chunks_before)\n )\n\n async with (\n await create_process(rhasspy, MIC_DOMAIN, mic_program)\n ) as mic_proc, (\n await create_process(rhasspy, VAD_DOMAIN, vad_program)\n ) as vad_proc:\n assert mic_proc.stdout is not None\n assert vad_proc.stdin is not None\n assert vad_proc.stdout is not None\n\n _LOGGER.info(\"Recording %s\", wav_path)\n mic_task = asyncio.create_task(async_read_event(mic_proc.stdout))\n vad_task = asyncio.create_task(async_read_event(vad_proc.stdout))\n pending = {mic_task, vad_task}\n\n before_command = True\n while True:\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n if mic_task in done:\n mic_event = mic_task.result()\n if mic_event is None:\n break\n\n # Process chunk\n if AudioChunk.is_type(mic_event.type):\n chunk = AudioChunk.from_event(mic_event)\n if is_first_chunk:\n _LOGGER.debug(\"Receiving audio\")\n is_first_chunk = False\n wav_file.setframerate(chunk.rate)\n wav_file.setsampwidth(chunk.width)\n wav_file.setnchannels(chunk.channels)\n\n await async_write_event(mic_event, vad_proc.stdin)\n if before_command:\n chunk_buffer.append(chunk)\n else:\n wav_file.writeframes(chunk.audio)\n\n # Next chunk\n mic_task = asyncio.create_task(\n async_read_event(mic_proc.stdout)\n )\n pending.add(mic_task)\n\n if vad_task in done:\n vad_event = vad_task.result()\n if vad_event is None:\n break\n\n if VoiceStarted.is_type(vad_event.type):\n if before_command:\n # Start of voice command\n voice_started = VoiceStarted.from_event(vad_event)\n if voice_started.timestamp is None:\n # Keep chunks before\n chunks_left = args.keep_chunks_before\n while chunk_buffer and (chunks_left > 0):\n chunk = chunk_buffer.popleft()\n wav_file.writeframes(chunk.audio)\n else:\n # Locate start chunk\n start_idx = 0\n for i, chunk in enumerate(chunk_buffer):\n if (chunk.timestamp is not None) and (\n chunk.timestamp >= voice_started.timestamp\n ):\n start_idx = i\n break\n\n # Back up by \"keep chunks\" and then write audio forward\n start_idx = max(\n 0, start_idx - args.keep_chunks_before\n )\n for i, chunk in enumerate(chunk_buffer):\n if i >= start_idx:\n wav_file.writeframes(chunk.audio)\n\n chunk_buffer.clear()\n\n before_command = False\n _LOGGER.info(\"Speaking started\")\n elif VoiceStopped.is_type(vad_event.type):\n # End of voice command\n _LOGGER.info(\"Speaking ended\")\n break\n\n # Next VAD event\n vad_task = asyncio.create_task(\n async_read_event(vad_proc.stdout)\n )\n pending.add(vad_task)\n\n # After chunks\n num_chunks_left = args.keep_chunks_after\n while num_chunks_left > 0:\n mic_event = await mic_task\n if mic_event is None:\n break\n\n if AudioChunk.is_type(mic_event.type):\n chunk = AudioChunk.from_event(mic_event)\n wav_file.writeframes(chunk.audio)\n num_chunks_left -= 1\n\n if num_chunks_left > 0:\n mic_task = asyncio.create_task(\n async_read_event(mic_proc.stdout)\n )\n\n\nif __name__ == \"__main__\":\n try:\n asyncio.run(main())\n except KeyboardInterrupt:\n pass\n" }, { "path": "bin/mic_test_energy.py", "content": "#!/usr/bin/env python3\n\"\"\"Prints microphone energy level to console for testing.\"\"\"\nimport argparse\nimport asyncio\nimport audioop\nimport logging\nfrom pathlib import Path\n\nfrom rhasspy3.audio import AudioChunk, AudioStop\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import async_read_event\nfrom rhasspy3.mic import DOMAIN as MIC_DOMAIN\nfrom rhasspy3.program import create_process\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--mic-program\", help=\"Name of mic program to use (overrides pipeline)\"\n )\n #\n parser.add_argument(\n \"--levels\", type=int, default=40, help=\"Number of levels to display\"\n )\n parser.add_argument(\n \"--numeric\",\n action=\"store_true\",\n help=\"Print energy numeric values instead of showing level\",\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n mic_program = args.mic_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not mic_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n mic_program = pipeline.mic\n\n assert mic_program, \"No mic program\"\n max_energy = 0\n\n async with (await create_process(rhasspy, MIC_DOMAIN, mic_program)) as mic_proc:\n assert mic_proc.stdout is not None\n\n while True:\n event = await async_read_event(mic_proc.stdout)\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n energy = -audioop.rms(chunk.audio, chunk.width)\n energy_bytes = bytes([energy & 0xFF, (energy >> 8) & 0xFF])\n debiased_energy = audioop.rms(\n audioop.add(\n chunk.audio,\n energy_bytes * (len(chunk.audio) // chunk.width),\n chunk.width,\n ),\n chunk.width,\n )\n\n max_energy = max(max_energy, debiased_energy)\n max_energy = max(1, max_energy)\n\n if args.numeric:\n # Print numbers\n print(debiased_energy, \"/\", max_energy)\n else:\n # Print graphic\n energy_level = int(args.levels * (debiased_energy / max_energy))\n energy_level = max(0, energy_level)\n print(\n \"\\r\", # We still use typewriters!\n \"[\",\n \"*\" * energy_level,\n \" \" * (args.levels - energy_level),\n \"]\",\n sep=\"\",\n end=\"\",\n )\n\n elif AudioStop.is_type(event.type):\n break\n\n\nif __name__ == \"__main__\":\n try:\n asyncio.run(main())\n except KeyboardInterrupt:\n pass\n" }, { "path": "bin/pipeline_run.py", "content": "#!/usr/bin/env python3\n\"\"\"Run a pipeline all or part of the way.\"\"\"\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport sys\nfrom pathlib import Path\nfrom typing import IO, Optional, Union\n\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.audio import DEFAULT_SAMPLES_PER_CHUNK\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import Event\nfrom rhasspy3.handle import Handled, NotHandled\nfrom rhasspy3.intent import Intent, NotRecognized\nfrom rhasspy3.pipeline import StopAfterDomain\nfrom rhasspy3.pipeline import run as run_pipeline\nfrom rhasspy3.wake import Detection\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n #\n parser.add_argument(\n \"--stop-after\",\n choices=[domain.value for domain in StopAfterDomain],\n help=\"Domain to stop pipeline after\",\n )\n #\n parser.add_argument(\n \"--wake-name\", help=\"Skip wake word detection and use name instead\"\n )\n parser.add_argument(\n \"--asr-wav\",\n help=\"Use WAV file for speech to text instead of mic input (skips wake)\",\n )\n parser.add_argument(\"--asr-text\", help=\"Use text for asr transcript (skips wake)\")\n parser.add_argument(\n \"--intent-json\", help=\"Use JSON for recognized intent (skips wake, asr)\"\n )\n parser.add_argument(\n \"--handle-text\", help=\"Use text for handle response (skips handle)\"\n )\n parser.add_argument(\n \"--tts-wav\", help=\"Play WAV file instead of text to speech response (skips tts)\"\n )\n\n parser.add_argument(\n \"--samples-per-chunk\", type=int, default=DEFAULT_SAMPLES_PER_CHUNK\n )\n parser.add_argument(\"--asr-chunks-to-buffer\", type=int, default=0)\n parser.add_argument(\"--loop\", action=\"store_true\", help=\"Keep pipeline running\")\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n wake_detection: Optional[Detection] = None\n if args.wake_name:\n # Wake word detection will be skipped\n wake_detection = Detection(name=args.wake_name)\n\n asr_wav_in: Optional[IO[bytes]] = None\n if args.asr_wav:\n # asr input will come from WAV file instead of mic\n asr_wav_in = open(args.asr_wav, \"rb\")\n\n asr_transcript: Optional[Transcript] = None\n if args.asr_text:\n # asr transcription will be skipped\n asr_transcript = Transcript(text=args.asr_text)\n\n intent_result: Optional[Union[Intent, NotRecognized]] = None\n if args.intent_json:\n # intent recognition will be skipped\n intent_event = Event.from_dict(json.loads(args.intent_json))\n if Intent.is_type(intent_event.type):\n intent_result = Intent.from_event(intent_event)\n elif NotRecognized.is_type(intent_event.type):\n intent_result = NotRecognized.from_event(intent_event)\n\n handle_result: Optional[Union[Handled, NotHandled]] = None\n if args.handle_text:\n # text/intent handling will be skipped\n handle_result = Handled(text=args.handle_text)\n\n tts_wav_in: Optional[IO[bytes]] = None\n if args.tts_wav:\n # tts synthesis will be skipped\n tts_wav_in = open(args.tts_wav, \"rb\")\n\n rhasspy = Rhasspy.load(args.config)\n\n while True:\n pipeline_result = await run_pipeline(\n rhasspy,\n args.pipeline,\n samples_per_chunk=args.samples_per_chunk,\n asr_chunks_to_buffer=args.asr_chunks_to_buffer,\n wake_detection=wake_detection,\n asr_wav_in=asr_wav_in,\n asr_transcript=asr_transcript,\n intent_result=intent_result,\n handle_result=handle_result,\n tts_wav_in=tts_wav_in,\n stop_after=args.stop_after,\n )\n\n json.dump(pipeline_result.to_dict(), sys.stdout, ensure_ascii=False)\n print(\"\")\n\n if not args.loop:\n break\n\n\nif __name__ == \"__main__\":\n try:\n asyncio.run(main())\n except KeyboardInterrupt:\n pass\n" }, { "path": "bin/program_download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport shlex\nimport string\nimport subprocess\nfrom pathlib import Path\n\nfrom rhasspy3.core import Rhasspy\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"domain\")\n parser.add_argument(\"program\")\n parser.add_argument(\"model\")\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n program_config = rhasspy.config.programs.get(args.domain, {}).get(args.program)\n assert program_config is not None, f\"No config for {args.domain} {args.program}\"\n\n install = program_config.install\n assert install is not None, f\"No install config for {args.domain} {args.program}\"\n\n downloads = install.downloads\n assert downloads is not None, f\"No downloads for {args.domain} {args.program}\"\n\n model = downloads.get(args.model)\n assert (\n model is not None\n ), f\"No download named {args.model} for {args.domain} {args.program}\"\n\n program_dir = rhasspy.programs_dir / args.domain / args.program\n data_dir = rhasspy.data_dir / args.domain / args.program\n\n default_mapping = {\n \"program_dir\": str(program_dir.absolute()),\n \"data_dir\": str(data_dir.absolute()),\n \"model\": str(args.model),\n }\n\n # Check if already installed\n if model.check_file is not None:\n check_file = Path(\n string.Template(model.check_file).safe_substitute(default_mapping)\n )\n if check_file.exists():\n _LOGGER.info(\"Installed: %s\", check_file)\n return\n\n download = install.download\n assert download is not None, f\"No download config for {args.domain} {args.program}\"\n\n download_command = string.Template(download.command).safe_substitute(\n default_mapping\n )\n _LOGGER.info(download_command)\n\n cwd = program_dir if program_dir.exists() else rhasspy.config_dir\n\n if download.shell:\n subprocess.check_call(download_command, shell=True, cwd=cwd)\n else:\n subprocess.check_call(shlex.split(download_command), cwd=cwd)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/program_install.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport shlex\nimport string\nimport subprocess\nfrom pathlib import Path\n\nfrom rhasspy3.core import Rhasspy\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"domain\")\n parser.add_argument(\"program\")\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n program_config = rhasspy.config.programs.get(args.domain, {}).get(args.program)\n assert program_config is not None, f\"No config for {args.domain} {args.program}\"\n\n install = program_config.install\n assert install is not None, f\"No install config for {args.domain} {args.program}\"\n\n program_dir = rhasspy.programs_dir / args.domain / args.program\n data_dir = rhasspy.data_dir / args.domain / args.program\n\n default_mapping = {\n \"program_dir\": str(program_dir.absolute()),\n \"data_dir\": str(data_dir.absolute()),\n }\n\n # Check if already installed\n if install.check_file is not None:\n check_file = Path(\n string.Template(install.check_file).safe_substitute(default_mapping)\n )\n if check_file.exists():\n _LOGGER.info(\"Installed: %s\", check_file)\n return\n\n install_command = string.Template(install.command).safe_substitute(default_mapping)\n _LOGGER.debug(install_command)\n\n cwd = program_dir if program_dir.exists() else rhasspy.config_dir\n\n if install.shell:\n subprocess.check_call(install_command, shell=True, cwd=cwd)\n else:\n subprocess.check_call(shlex.split(install_command), cwd=cwd)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/satellite_run.py", "content": "#!/usr/bin/env python3\n\"\"\"Run satellite loop.\"\"\"\nimport argparse\nimport asyncio\nimport logging\nfrom collections import deque\nfrom pathlib import Path\nfrom typing import Deque, List\n\nfrom rhasspy3.audio import AudioChunk, AudioStop\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import Event, async_read_event, async_write_event\nfrom rhasspy3.mic import DOMAIN as MIC_DOMAIN\nfrom rhasspy3.program import create_process\nfrom rhasspy3.remote import DOMAIN as REMOTE_DOMAIN\nfrom rhasspy3.snd import DOMAIN as SND_DOMAIN\nfrom rhasspy3.snd import Played\nfrom rhasspy3.wake import detect\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-s\", \"--satellite\", default=\"default\", help=\"Name of satellite to use\"\n )\n #\n parser.add_argument(\n \"--mic-program\",\n help=\"Program to use for mic input (overrides satellite)\",\n )\n parser.add_argument(\n \"--wake-program\",\n help=\"Program to use for wake word detection (overiddes satellite)\",\n )\n parser.add_argument(\n \"--remote-program\",\n help=\"Program to use for remote communication with base station (overrides satellite)\",\n )\n parser.add_argument(\n \"--snd-program\",\n help=\"Program to use for audio output (overrides satellite)\",\n )\n #\n parser.add_argument(\"--asr-chunks-to-buffer\", type=int, default=0)\n #\n parser.add_argument(\"--loop\", action=\"store_true\", help=\"Keep satellite running\")\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n mic_program = args.mic_program\n wake_program = args.wake_program\n remote_program = args.remote_program\n snd_program = args.snd_program\n satellite = rhasspy.config.satellites.get(args.satellite)\n\n if not mic_program:\n assert satellite is not None, f\"No satellite named {args.satellite}\"\n mic_program = satellite.mic\n\n assert mic_program, \"No mic program\"\n\n if not wake_program:\n assert satellite is not None, f\"No satellite named {args.satellite}\"\n wake_program = satellite.wake\n\n assert wake_program, \"No wake program\"\n\n if not remote_program:\n assert satellite is not None, f\"No satellite named {args.satellite}\"\n remote_program = satellite.remote\n\n assert remote_program, \"No remote program\"\n\n if not snd_program:\n assert satellite is not None, f\"No satellite named {args.satellite}\"\n snd_program = satellite.snd\n\n assert snd_program, \"No snd program\"\n\n while True:\n chunk_buffer: Deque[Event] = deque(maxlen=args.asr_chunks_to_buffer)\n snd_buffer: List[Event] = []\n\n async with (await create_process(rhasspy, MIC_DOMAIN, mic_program)) as mic_proc:\n assert mic_proc.stdout is not None\n\n detection = await detect(\n rhasspy, wake_program, mic_proc.stdout, chunk_buffer\n )\n if detection is None:\n continue\n\n async with (\n await create_process(rhasspy, REMOTE_DOMAIN, remote_program)\n ) as remote_proc:\n assert remote_proc.stdin is not None\n assert remote_proc.stdout is not None\n\n while chunk_buffer:\n await async_write_event(chunk_buffer.pop(), remote_proc.stdin)\n\n mic_task = asyncio.create_task(async_read_event(mic_proc.stdout))\n remote_task = asyncio.create_task(async_read_event(remote_proc.stdout))\n pending = {mic_task, remote_task}\n\n try:\n # Stream to remote until audio is received\n while True:\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n\n if mic_task in done:\n mic_event = mic_task.result()\n if mic_event is None:\n break\n\n if AudioChunk.is_type(mic_event.type):\n await async_write_event(mic_event, remote_proc.stdin)\n\n mic_task = asyncio.create_task(\n async_read_event(mic_proc.stdout)\n )\n pending.add(mic_task)\n\n if remote_task in done:\n remote_event = remote_task.result()\n if remote_event is not None:\n snd_buffer.append(remote_event)\n\n for task in pending:\n task.cancel()\n\n break\n\n # Output audio\n async with (\n await create_process(rhasspy, SND_DOMAIN, snd_program)\n ) as snd_proc:\n assert snd_proc.stdin is not None\n assert snd_proc.stdout is not None\n\n for remote_event in snd_buffer:\n if AudioChunk.is_type(remote_event.type):\n await async_write_event(remote_event, snd_proc.stdin)\n elif AudioStop.is_type(remote_event.type):\n # Unexpected, but it could happen\n continue\n\n while True:\n remote_event = await async_read_event(remote_proc.stdout)\n if remote_event is None:\n break\n\n if AudioChunk.is_type(remote_event.type):\n await async_write_event(remote_event, snd_proc.stdin)\n elif AudioStop.is_type(remote_event.type):\n await async_write_event(remote_event, snd_proc.stdin)\n break\n\n # Wait for audio to finish playing\n while True:\n snd_event = await async_read_event(snd_proc.stdout)\n if snd_event is None:\n break\n\n if Played.is_type(snd_event.type):\n break\n except Exception:\n _LOGGER.exception(\n \"Unexpected error communicating with remote base station\"\n )\n\n if not args.loop:\n break\n\n\nif __name__ == \"__main__\":\n try:\n asyncio.run(main())\n except KeyboardInterrupt:\n pass\n" }, { "path": "bin/server_run.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport os\nimport shlex\nimport string\nimport subprocess\nimport sys\nfrom pathlib import Path\nfrom typing import List, Union\n\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.util import merge_dict\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\"domain\", help=\"Domain of server (asr, tts, etc.)\")\n parser.add_argument(\"server\", help=\"Name of server to run\")\n\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n server = rhasspy.config.servers[args.domain][args.server]\n\n program_dir = rhasspy.programs_dir / args.domain / args.server\n data_dir = rhasspy.data_dir / args.domain / args.server\n\n # ${variables} available within command and template args\n default_mapping = {\n \"program_dir\": str(program_dir.absolute()),\n \"data_dir\": str(data_dir.absolute()),\n }\n\n command_str = server.command\n command_mapping = dict(default_mapping)\n\n if server.template_args:\n # Substitute within template args\n args_mapping = dict(server.template_args)\n for arg_name, arg_str in args_mapping.items():\n if not isinstance(arg_str, str):\n continue\n\n arg_template = string.Template(arg_str)\n args_mapping[arg_name] = arg_template.safe_substitute(default_mapping)\n\n merge_dict(command_mapping, args_mapping)\n\n command_template = string.Template(command_str)\n command_str = command_template.safe_substitute(command_mapping)\n\n env = dict(os.environ)\n\n # Add rhasspy3/bin to $PATH\n env[\"PATH\"] = f'{rhasspy.base_dir}/bin:${env[\"PATH\"]}'\n\n # Ensure stdout is flushed for Python programs\n env[\"PYTHONUNBUFFERED\"] = \"1\"\n\n server_dir = rhasspy.programs_dir / args.domain / args.server\n cwd = server_dir if server_dir.is_dir() else rhasspy.base_dir\n\n if server.shell:\n command: Union[str, List[str]] = command_str\n else:\n command = shlex.split(command_str)\n\n _LOGGER.debug(command)\n proc = subprocess.Popen(command, shell=server.shell, cwd=cwd, env=env)\n with proc:\n sys.exit(proc.wait())\n\n\nif __name__ == \"__main__\":\n try:\n main()\n except KeyboardInterrupt:\n pass\n" }, { "path": "bin/snd_adapter_raw.py", "content": "#!/usr/bin/env python3\n\"\"\"Play audio through a command that accepts raw PCM.\"\"\"\nimport argparse\nimport logging\nimport shlex\nimport subprocess\nfrom pathlib import Path\n\nfrom rhasspy3.audio import (\n DEFAULT_OUT_CHANNELS,\n DEFAULT_OUT_RATE,\n DEFAULT_OUT_WIDTH,\n AudioChunk,\n AudioChunkConverter,\n AudioStop,\n)\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.snd import Played\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\n \"--rate\", type=int, default=DEFAULT_OUT_RATE, help=\"Sample rate (hertz)\"\n )\n parser.add_argument(\n \"--width\", type=int, default=DEFAULT_OUT_WIDTH, help=\"Sample width (bytes)\"\n )\n parser.add_argument(\n \"--channels\",\n type=int,\n default=DEFAULT_OUT_CHANNELS,\n help=\"Sample channel count\",\n )\n parser.add_argument(\"--shell\", action=\"store_true\", help=\"Run command with shell\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n if args.shell:\n command = args.command\n else:\n command = shlex.split(args.command)\n\n try:\n proc = subprocess.Popen(\n command, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL\n )\n assert proc.stdin is not None\n\n converter = AudioChunkConverter(args.rate, args.width, args.channels)\n with proc:\n while True:\n event = read_event()\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n chunk = converter.convert(chunk)\n proc.stdin.write(chunk.audio)\n proc.stdin.flush()\n elif AudioStop.is_type(event.type):\n break\n finally:\n write_event(Played().event())\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/snd_play.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport asyncio\nimport logging\nimport os\nimport sys\nfrom pathlib import Path\n\nfrom rhasspy3.audio import DEFAULT_SAMPLES_PER_CHUNK\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.snd import play\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"wav_file\", nargs=\"*\", help=\"Path to WAV file(s) to play\")\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline\")\n parser.add_argument(\"--snd-program\", help=\"Audio output program name\")\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=DEFAULT_SAMPLES_PER_CHUNK,\n help=\"Samples to send to snd program at a time\",\n )\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n snd_program = args.snd_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not snd_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n snd_program = pipeline.snd\n\n assert snd_program, \"No snd program\"\n\n if args.wav_file:\n for wav_path in args.wav_file:\n with open(wav_path, \"rb\") as wav_file:\n await play(rhasspy, snd_program, wav_file, args.samples_per_chunk)\n else:\n if os.isatty(sys.stdin.fileno()):\n print(\"Reading WAV data from stdin\", file=sys.stderr)\n\n await play(\n rhasspy,\n snd_program,\n sys.stdin.buffer,\n args.samples_per_chunk,\n )\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/tts_adapter_http.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport wave\nfrom pathlib import Path\nfrom urllib.parse import urlencode\nfrom urllib.request import urlopen\n\nfrom rhasspy3.audio import DEFAULT_SAMPLES_PER_CHUNK, AudioChunk, AudioStart, AudioStop\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.tts import Synthesize\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"url\",\n help=\"URL of API endpoint\",\n )\n parser.add_argument(\n \"--param\",\n nargs=2,\n action=\"append\",\n metavar=(\"name\", \"value\"),\n help=\"Name/value of query parameter\",\n )\n #\n parser.add_argument(\n \"--samples-per-chunk\", type=int, default=DEFAULT_SAMPLES_PER_CHUNK\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n params = {}\n if args.param:\n for key, value in params.items():\n # Don't include empty parameters\n if value:\n params[key] = value\n\n try:\n while True:\n event = read_event()\n if event is None:\n break\n\n if Synthesize.is_type(event.type):\n synthesize = Synthesize.from_event(event)\n\n params[\"text\"] = synthesize.text\n url = args.url + \"?\" + urlencode(params)\n\n with urlopen(url) as response:\n with wave.open(response, \"rb\") as wav_file:\n rate = wav_file.getframerate()\n width = wav_file.getsampwidth()\n channels = wav_file.getnchannels()\n\n num_frames = wav_file.getnframes()\n audio_bytes = wav_file.readframes(num_frames)\n\n bytes_per_chunk = args.samples_per_chunk * width\n timestamp = 0\n write_event(\n AudioStart(rate, width, channels, timestamp=timestamp).event()\n )\n while audio_bytes:\n chunk = AudioChunk(\n rate,\n width,\n channels,\n audio_bytes[:bytes_per_chunk],\n timestamp=timestamp,\n )\n write_event(chunk.event())\n timestamp += chunk.milliseconds\n audio_bytes = audio_bytes[bytes_per_chunk:]\n\n write_event(AudioStop(timestamp=timestamp).event())\n except KeyboardInterrupt:\n pass\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/tts_adapter_text2wav.py", "content": "#!/usr/bin/env python3\n\"\"\"\nRuns a text to speech command that returns WAV audio on stdout or in a temp file.\n\"\"\"\nimport argparse\nimport io\nimport logging\nimport shlex\nimport subprocess\nimport tempfile\nimport wave\nfrom pathlib import Path\n\nfrom rhasspy3.audio import DEFAULT_SAMPLES_PER_CHUNK, AudioChunk, AudioStart, AudioStop\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.tts import Synthesize\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\n \"--temp_file\",\n action=\"store_true\",\n help=\"Command has {temp_file} and will write output to it\",\n )\n parser.add_argument(\n \"--text\",\n action=\"store_true\",\n help=\"Command has {text} argument\",\n )\n #\n parser.add_argument(\n \"--samples-per-chunk\", type=int, default=DEFAULT_SAMPLES_PER_CHUNK\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n try:\n while True:\n event = read_event()\n if event is None:\n break\n\n if Synthesize.is_type(event.type):\n synthesize = Synthesize.from_event(event)\n wav_bytes = text_to_wav(args, synthesize.text)\n with io.BytesIO(wav_bytes) as wav_io:\n with wave.open(wav_io, \"rb\") as wav_file:\n rate = wav_file.getframerate()\n width = wav_file.getsampwidth()\n channels = wav_file.getnchannels()\n\n num_frames = wav_file.getnframes()\n audio_bytes = wav_file.readframes(num_frames)\n\n bytes_per_chunk = args.samples_per_chunk * width\n timestamp = 0\n write_event(\n AudioStart(rate, width, channels, timestamp=timestamp).event()\n )\n while audio_bytes:\n chunk = AudioChunk(\n rate,\n width,\n channels,\n audio_bytes[:bytes_per_chunk],\n timestamp=timestamp,\n )\n write_event(chunk.event())\n timestamp += chunk.milliseconds\n audio_bytes = audio_bytes[bytes_per_chunk:]\n\n write_event(AudioStop(timestamp=timestamp).event())\n except KeyboardInterrupt:\n pass\n\n\ndef text_to_wav(args: argparse.Namespace, text: str) -> bytes:\n command_str = args.command\n format_args = {}\n if args.text:\n format_args[\"text\"] = text\n text = \"\" # Pass as arg instead\n\n if args.temp_file:\n with tempfile.NamedTemporaryFile(mode=\"wb\", suffix=\".wav\") as wav_file:\n format_args[\"temp_file\"] = wav_file.name\n command_str = command_str.format(**format_args)\n command = shlex.split(command_str)\n\n # Send stdout to devnull so it doesn't interfere with our events\n subprocess.run(\n command, check=True, stdout=subprocess.DEVNULL, input=text.encode()\n )\n wav_file.seek(0)\n return Path(wav_file.name).read_bytes()\n\n else:\n command_str = command_str.format(**format_args)\n command = shlex.split(command_str)\n return subprocess.check_output(command, input=text.encode())\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/tts_speak.py", "content": "#!/usr/bin/env python3\n\"\"\"Synthesize and speak audio.\"\"\"\nimport argparse\nimport asyncio\nimport io\nimport json\nimport logging\nimport os\nimport sys\nfrom pathlib import Path\n\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.snd import play\nfrom rhasspy3.tts import synthesize\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"text\", nargs=\"*\", help=\"Text to speak (default: stdin)\")\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline\")\n parser.add_argument(\"--tts-program\", help=\"TTS program name\")\n parser.add_argument(\"--snd-program\", help=\"Audio output program name\")\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=1024,\n help=\"Samples to send to snd program at a time\",\n )\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n tts_program = args.tts_program\n snd_program = args.snd_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not tts_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n tts_program = pipeline.tts\n\n if not snd_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n snd_program = pipeline.snd\n\n assert tts_program, \"No tts program\"\n assert snd_program, \"No snd program\"\n\n if args.text:\n lines = args.text\n else:\n lines = sys.stdin\n if os.isatty(sys.stdin.fileno()):\n print(\"Reading text from stdin\", file=sys.stderr)\n\n for line in lines:\n line = line.strip()\n if not line:\n continue\n\n with io.BytesIO() as wav_io:\n await synthesize(rhasspy, tts_program, line, wav_io)\n wav_io.seek(0)\n play_result = await play(\n rhasspy, snd_program, wav_io, args.samples_per_chunk\n )\n if play_result is not None:\n json.dump(play_result.event().to_dict(), sys.stdout, ensure_ascii=False)\n print(\"\", flush=True)\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/tts_synthesize.py", "content": "#!/usr/bin/env python3\n\"\"\"Synthesize WAV audio from text.\"\"\"\nimport argparse\nimport asyncio\nimport io\nimport logging\nimport sys\nfrom pathlib import Path\n\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.tts import synthesize\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"text\", help=\"Text to speak\")\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline\")\n parser.add_argument(\"--tts-program\", help=\"TTS program name\")\n parser.add_argument(\"-f\", \"--file\", help=\"Write to file instead of stdout\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n tts_program = args.tts_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not tts_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n tts_program = pipeline.tts\n\n assert tts_program, \"No tts program\"\n\n with io.BytesIO() as wav_out:\n await synthesize(rhasspy, tts_program, args.text, wav_out)\n wav_bytes = wav_out.getvalue()\n\n if args.file:\n output_path = Path(args.file)\n output_path.parent.mkdir(parents=True, exist_ok=True)\n output_path.write_bytes(wav_bytes)\n else:\n sys.stdout.buffer.write(wav_bytes)\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/vad_adapter_raw.py", "content": "#!/usr/bin/env python3\n\"\"\"Voice activity detection programs that accept raw PCM audio and print a speech probability for each chunk.\"\"\"\nimport argparse\nimport logging\nimport shlex\nimport subprocess\nimport time\nfrom pathlib import Path\nfrom typing import Optional\n\nfrom rhasspy3.audio import AudioChunk, AudioChunkConverter, AudioStop\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.vad import Segmenter, VoiceStarted, VoiceStopped\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n #\n parser.add_argument(\n \"--rate\",\n type=int,\n required=True,\n help=\"Sample rate (hz)\",\n )\n parser.add_argument(\n \"--width\",\n type=int,\n required=True,\n help=\"Sample width bytes\",\n )\n parser.add_argument(\n \"--channels\",\n type=int,\n required=True,\n help=\"Sample channel count\",\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n required=True,\n type=int,\n help=\"Samples to send to command at a time\",\n )\n #\n parser.add_argument(\n \"--threshold\",\n type=float,\n default=0.5,\n help=\"Speech probability threshold (0-1)\",\n )\n parser.add_argument(\n \"--speech-seconds\",\n type=float,\n default=0.3,\n )\n parser.add_argument(\n \"--silence-seconds\",\n type=float,\n default=0.5,\n )\n parser.add_argument(\n \"--timeout-seconds\",\n type=float,\n default=15.0,\n )\n parser.add_argument(\n \"--reset-seconds\",\n type=float,\n default=1,\n )\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n bytes_per_chunk = args.samples_per_chunk * args.width * args.channels\n seconds_per_chunk = args.samples_per_chunk / args.rate\n\n command = shlex.split(args.command)\n with subprocess.Popen(\n command, stdin=subprocess.PIPE, stdout=subprocess.PIPE\n ) as proc:\n assert proc.stdin is not None\n assert proc.stdout is not None\n\n segmenter = Segmenter(\n args.speech_seconds,\n args.silence_seconds,\n args.timeout_seconds,\n args.reset_seconds,\n )\n converter = AudioChunkConverter(args.rate, args.width, args.channels)\n audio_bytes = bytes()\n is_first_audio = True\n sent_started = False\n sent_stopped = False\n last_stop_timestamp: Optional[int] = None\n\n while True:\n event = read_event()\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n if is_first_audio:\n _LOGGER.debug(\"Receiving audio\")\n is_first_audio = False\n\n chunk = AudioChunk.from_event(event)\n chunk = converter.convert(chunk)\n audio_bytes += chunk.audio\n timestamp = (\n time.monotonic_ns() if chunk.timestamp is None else chunk.timestamp\n )\n last_stop_timestamp = timestamp + chunk.milliseconds\n\n # Handle uneven chunk sizes\n while len(audio_bytes) >= bytes_per_chunk:\n chunk_bytes = audio_bytes[:bytes_per_chunk]\n proc.stdin.write(chunk_bytes)\n proc.stdin.flush()\n\n line = proc.stdout.readline().decode()\n if line:\n speech_probability = float(line)\n is_speech = speech_probability > args.threshold\n segmenter.process(\n chunk=chunk_bytes,\n chunk_seconds=seconds_per_chunk,\n is_speech=is_speech,\n timestamp=timestamp,\n )\n\n if (not sent_started) and segmenter.started:\n _LOGGER.debug(\"Voice started\")\n write_event(\n VoiceStarted(\n timestamp=segmenter.start_timestamp\n ).event()\n )\n sent_started = True\n\n if (not sent_stopped) and segmenter.stopped:\n if segmenter.timeout:\n _LOGGER.info(\"Voice timeout\")\n else:\n _LOGGER.debug(\"Voice stopped\")\n\n write_event(\n VoiceStopped(timestamp=segmenter.stop_timestamp).event()\n )\n sent_stopped = True\n\n audio_bytes = audio_bytes[bytes_per_chunk:]\n\n elif AudioStop.is_type(event.type):\n _LOGGER.debug(\"Audio stopped\")\n if not sent_stopped:\n write_event(VoiceStopped(timestamp=last_stop_timestamp).event())\n sent_stopped = True\n\n proc.stdin.close()\n break\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/vad_segment_wav.py", "content": "#!/usr/bin/env python3\n\"\"\"Prints voice start/stop in WAV file.\"\"\"\nimport argparse\nimport asyncio\nimport io\nimport json\nimport logging\nimport sys\nimport time\nimport wave\nfrom pathlib import Path\nfrom typing import Iterable, Optional\n\nfrom rhasspy3.audio import DEFAULT_SAMPLES_PER_CHUNK, AudioChunk, AudioStart, AudioStop\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import async_read_event, async_write_event\nfrom rhasspy3.program import create_process\nfrom rhasspy3.vad import DOMAIN, VoiceStarted, VoiceStopped\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--vad-program\", help=\"Name of vad program to use (overrides pipeline)\"\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=DEFAULT_SAMPLES_PER_CHUNK,\n help=\"Samples to process at a time\",\n )\n parser.add_argument(\"wav\", nargs=\"*\", help=\"Path(s) to WAV file(s)\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n vad_program = args.vad_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not vad_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n vad_program = pipeline.vad\n\n assert vad_program, \"No vad program\"\n\n async with (await create_process(rhasspy, DOMAIN, vad_program)) as vad_proc:\n assert vad_proc.stdin is not None\n assert vad_proc.stdout is not None\n\n for wav_bytes in get_wav_bytes(args):\n with io.BytesIO(wav_bytes) as wav_io:\n with wave.open(wav_io, \"rb\") as wav_file:\n rate = wav_file.getframerate()\n width = wav_file.getsampwidth()\n channels = wav_file.getnchannels()\n\n timestamp = 0\n await async_write_event(\n AudioStart(rate, width, channels, timestamp=timestamp).event(),\n vad_proc.stdin,\n )\n\n audio_bytes = wav_file.readframes(args.samples_per_chunk)\n while audio_bytes:\n chunk = AudioChunk(\n rate, width, channels, audio_bytes, timestamp=timestamp\n )\n await async_write_event(\n chunk.event(),\n vad_proc.stdin,\n )\n timestamp += chunk.milliseconds\n audio_bytes = wav_file.readframes(args.samples_per_chunk)\n\n await async_write_event(\n AudioStop(timestamp=timestamp).event(), vad_proc.stdin\n )\n\n voice_started: Optional[VoiceStarted] = None\n voice_stopped: Optional[VoiceStopped] = None\n while True:\n event = await async_read_event(vad_proc.stdout)\n if event is None:\n break\n\n if VoiceStarted.is_type(event.type):\n voice_started = VoiceStarted.from_event(event)\n if voice_started.timestamp is None:\n voice_started.timestamp = time.monotonic_ns()\n\n _LOGGER.debug(voice_started)\n json.dump(\n voice_started.event().to_dict(), sys.stdout, ensure_ascii=False\n )\n print(\"\", flush=True)\n elif VoiceStopped.is_type(event.type):\n voice_stopped = VoiceStopped.from_event(event)\n if voice_stopped.timestamp is None:\n voice_stopped.timestamp = time.monotonic_ns()\n\n _LOGGER.debug(voice_stopped)\n json.dump(\n voice_stopped.event().to_dict(), sys.stdout, ensure_ascii=False\n )\n print(\"\", flush=True)\n break\n\n\ndef get_wav_bytes(args: argparse.Namespace) -> Iterable[bytes]:\n \"\"\"Yields WAV audio from stdin or args.\"\"\"\n if args.wav:\n # WAV file path(s)\n for wav_path in args.wav:\n with open(wav_path, \"rb\") as wav_file:\n yield wav_file.read()\n else:\n # WAV on stdin\n yield sys.stdin.buffer.read()\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "bin/wake_adapter_raw.py", "content": "#!/usr/bin/env python3\n\"\"\"Wake word detection with a command that accepts raw PCM audio and prints a line for each detection.\"\"\"\nimport argparse\nimport logging\nimport shlex\nimport subprocess\nimport threading\nimport time\nfrom dataclasses import dataclass\nfrom pathlib import Path\nfrom typing import IO\n\nfrom rhasspy3.audio import AudioChunk, AudioStop\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.wake import Detection, NotDetected\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\n@dataclass\nclass State:\n timestamp: int = 0\n detected: bool = False\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"command\",\n help=\"Command to run\",\n )\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n command = shlex.split(args.command)\n with subprocess.Popen(\n command, stdin=subprocess.PIPE, stdout=subprocess.PIPE\n ) as proc:\n assert proc.stdin is not None\n assert proc.stdout is not None\n\n state = State()\n threading.Thread(target=write_proc, args=(proc.stdout, state)).start()\n\n while not state.detected:\n event = read_event()\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n state.timestamp = (\n chunk.timestamp\n if chunk.timestamp is not None\n else time.monotonic_ns()\n )\n proc.stdin.write(chunk.audio)\n proc.stdin.flush()\n elif AudioStop.is_type(event.type):\n proc.stdin.close()\n break\n\n if not state.detected:\n write_event(NotDetected().event())\n\n\ndef write_proc(reader: IO[bytes], state: State):\n try:\n for line in reader:\n line = line.strip()\n if line:\n write_event(\n Detection(name=line.decode(), timestamp=state.timestamp).event()\n )\n state.detected = True\n break\n except Exception:\n _LOGGER.exception(\"Unexpected error in write thread\")\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "bin/wake_detect.py", "content": "#!/usr/bin/env python3\n\"\"\"Wait for wake word to be detected.\"\"\"\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport sys\nfrom pathlib import Path\n\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.mic import DOMAIN as MIC_DOMAIN\nfrom rhasspy3.program import create_process\nfrom rhasspy3.wake import detect\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"-p\", \"--pipeline\", default=\"default\", help=\"Name of pipeline to use\"\n )\n parser.add_argument(\n \"--mic-program\", help=\"Name of mic program to use (overrides pipeline)\"\n )\n parser.add_argument(\n \"--wake-program\", help=\"Name of wake program to use (overrides pipeline)\"\n )\n #\n parser.add_argument(\"--loop\", action=\"store_true\", help=\"Keep detecting wake words\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n mic_program = args.mic_program\n wake_program = args.wake_program\n pipeline = rhasspy.config.pipelines.get(args.pipeline)\n\n if not mic_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n mic_program = pipeline.mic\n\n assert mic_program, \"No mic program\"\n _LOGGER.debug(\"mic program: %s\", mic_program)\n\n if not wake_program:\n assert pipeline is not None, f\"No pipeline named {args.pipeline}\"\n wake_program = pipeline.wake\n\n assert wake_program, \"No wake program\"\n _LOGGER.debug(\"wake program: %s\", wake_program)\n\n # Detect wake word\n while True:\n async with (await create_process(rhasspy, MIC_DOMAIN, mic_program)) as mic_proc:\n assert mic_proc.stdout is not None\n _LOGGER.debug(\"Detecting wake word\")\n detection = await detect(rhasspy, wake_program, mic_proc.stdout)\n if detection is not None:\n json.dump(detection.event().to_dict(), sys.stdout, ensure_ascii=False)\n print(\"\", flush=True)\n\n if not args.loop:\n break\n\n\nif __name__ == \"__main__\":\n try:\n asyncio.run(main())\n except KeyboardInterrupt:\n pass\n" }, { "path": "docs/README.md", "content": "# Rhasspy 3\n\n* [Tutorial](tutorial.md)\n* [Domains](domains.md)\n* [Wyoming Protcol](wyoming.md)\n* [Adapters](adapters.md)\n" }, { "path": "docs/adapters.md", "content": "# Adapters\n\nScripts in `bin/`:\n\n* `asr_adapter_raw2text.py`\n * Raw audio stream in, text or JSON out\n* `asr_adapter_wav2text.py`\n * WAV file(s) in, text or JSON out (per file)\n* `handle_adapter_json.py`\n * Intent JSON in, text response out\n* `handle_adapter_text.py`\n * Transcription in, text response out\n* `mic_adapter_raw.py`\n * Raw audio stream in\n* `snd_adapter_raw.py`\n * Raw audio stream out\n* `tts_adapter_http.py`\n * HTTP POST to endpoint with text, WAV out\n* `tts_adapter_text2wav.py`\n * Text in, WAV out\n* `vad_adapter_raw.py`\n * Raw audio stream in, speech probability out (one line per chunk)\n* `wake_adapter_raw.py`\n * Raw audio stream in, name of detected model out (one line per detection)\n* `client_unix_socket.py`\n * Send/receive events over Unix domain socket\n\n\n![Wyoming protocol adapter](img/adapter.png)\n" }, { "path": "docs/domains.md", "content": "# Domains\n\nPrograms belong to a specific domain. This defines the kinds of [events](wyoming.md) they are expected to receive and emit.\n\n## mic\n\nEmits `audio-chunk` events, ideally with a `timestamp`.\n\n\n## wake\n\nReceives `audio-chunk` events.\nEmits `detection` event(s) or a `not-detected` event if the program exits without a detection.\n\n\n## asr\n\nReceives an `audio-start` event, followed by zero or more `audio-chunk` events.\n\nAn `audio-stop` event must trigger a `transcript` event to be emitted.\n\n\n## vad\n\nReceives `audio-chunk` events.\n\nEmits `voice-started` with the `timestamp` of the `audio-chunk` when the user started speaking.\n\nEmits `voice-stopped` with the `timestamp` of the `audio-chunk` when the user finished speaking.\n\n\n## intent\n\nOptional. The `handle` domain can handle `transcript` events directly.\n\nReceives `recognize` events.\n\nEmits either an `intent` or a `not-recognized` event.\n\n\n## handle\n\nReceives one of the following event types: `transcript`, `intent`, or `not-recognized`.\n\nEmits either a `handle` or `not-handled` event.\n\n\n## tts\n\nReceives a `synthesize` event.\n\nEmits an `audio-start` event followed by zero or more `audio-chunk` events, and then an `audio-stop` event.\n\n\n## snd\n\nReceives `audio-chunk` events until an `audio-stop` event.\n\nMust emit `played` event when audio has finished playing.\n" }, { "path": "docs/home_assistant.md", "content": "# Home Assistant\n\nThis will connect Rhasspy to Home Assistant via [Assist](https://www.home-assistant.io/docs/assist).\n\nInstall the Home Assistant intent handler:\n\n```sh\nmkdir -p config/programs/handle/\ncp -R programs/handle/home_assistant config/programs/handle/\n```\n\nCreate a long-lived access token in Home Assistant (inside your profile):\n\n![Long-lived access token](img/ha_token.png)\n\nCopy the **entire** access token (with CTRL+A, not just selecting what you can see) and put it in the data directory:\n\n```sh\nmkdir -p config/data/handle/home_assistant/\necho \"MY-LONG-LIVED-ACCESS-TOKEN\" > config/data/handle/home_assistant/token\n```\n\nAdd to your `configuration.yaml`:\n\n\n```yaml\nprograms:\n handle:\n home_assistant:\n command: |\n bin/converse.py --language \"${language}\" \"${url}\" \"${token_file}\"\n adapter: |\n handle_adapter_text.py\n template_args:\n url: \"http://localhost:8123/api/conversation/process\"\n token_file: \"${data_dir}/token\"\n language: \"en\"\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n handle:\n name: home_assistant\n tts: ...\n snd: ...\n```\n\nMake sure your Home Assistant server is running, and test out a command:\n\n```sh\nscript/run bin/handle_text.py \"Turn on the bed light\"\n```\n\nReplace \"bed light\" with the name of a device you have connected to Home Assistant.\n\nIf successful, you should see JSON printed with the response text, like:\n\n```sh\n{\"type\": \"handled\", \"data\": {\"text\": \"Turned on light\"}}\n```\n\nThis also works over HTTP:\n\n```sh\ncurl -X POST --data 'Turn on the bed light' 'localhost:13331/handle/handle'\n```\n\nNow you can run your full pipeline and control Home Assistant!\n" }, { "path": "docs/satellite.md", "content": "# Satellite\n\nOnce you have a Rhasspy HTTP server running, you can use Rhasspy as a satellite on a separate device.\n\n**NOTE:** Rhasspy satellites do not need to run Python or any Rhasspy software. They can use the websocket API directly, or talk directly to a running pipeline.\n\nOn your satellite, clone the repo:\n\n```sh\ngit clone https://github.com/rhasspy/rhasspy3\ncd rhasspy3\n```\n\nInstall the websocket utility:\n\n```sh\nmkdir -p config/programs/remote/\ncp -R programs/remote/websocket config/programs/remote/\nconfig/programs/remote/websocket/script/setup\n```\n\nInstall [Porcupine](https://github.com/Picovoice/porcupine):\n\n```sh\nmkdir -p config/programs/wake/\ncp -R programs/wake/porcupine1 config/programs/wake/\nconfig/programs/wake/porcupine1/script/setup\n```\n\nCheck available wake word models by running \n\n```sh\nconfig/programs/wake/porcupine1/script/list_models\n```\n\nand choose one. We'll use \"porcupine_linux.ppn\" as an example, but this will be **different on a Raspberry Pi**.\n\nNext, create `config/configuration.yaml` with:\n\n```yaml\nprograms:\n mic:\n arecord:\n command: |\n arecord -q -r 16000 -c 1 -f S16_LE -t raw -\n adapter: |\n mic_adapter_raw.py --samples-per-chunk 1024 --rate 16000 --width 2 --channels 1\n\n wake:\n porcupine1:\n command: |\n .venv/bin/python3 bin/porcupine_stream.py --model \"${model}\"\n template_args:\n model: \"porcupine_linux.ppn\"\n\n remote:\n websocket:\n command: |\n script/run \"${uri}\"\n template_args:\n uri: \"ws://localhost:13331/pipeline/asr-tts\"\n\nsatellites:\n default:\n mic:\n name: arecord\n wake:\n name: porcupine1\n remote:\n name: websocket\n snd:\n name: aplay\n```\n\nReplace the model in `porcupine1` with your selection, and adjust the URI in `websocket` to point to your Rhasspy server.\n\nNow you can run your satellite:\n\n```sh\nscript/run bin/satellite_run.py --debug --loop\n```\n\n(say \"porcupine\", *pause*, say voice command, *wait*)\n\nIf everything is working, you should hear a response being spoken. Press CTRL+C to quit.\n" }, { "path": "docs/tutorial.md", "content": "# Tutorial\n\nWelcome to Rhasspy 3! This is a developer preview, so many of the manual steps here will be replaced with something more user-friendly in the future.\n\n\n## Installing Rhasspy 3\n\nTo get started, just clone the repo. Rhasspy's core does not currently have any dependencies outside the Python standard library.\n\n```sh\ngit clone https://github.com/rhasspy/rhasspy3\ncd rhasspy3\n```\n\n\n## Layout\n\nInstalled programs and downloaded models are stored in the `config` directory, which is empty by default:\n\n* `rhasspy3/config/`\n * `configuration.yaml` - overrides `rhasspy3/configuration.yaml`\n * `programs/` - installed programs\n * `/`\n * `/`\n * `data/` - downloaded models\n * `/`\n * `/`\n \nPrograms in Rhasspy are divided into [domains](domains.md).\n\n\n## Configuration\n\nRhasspy loads two configuration files:\n\n1. `rhasspy3/configuration.yaml` (base)\n2. `config/configuration.yaml` (user)\n\nThe file in `config` will override the base configuration. You can see what the final configuration looks like with:\n\n```sh\nscript/run bin/config_print.py\n```\n\n\n## Microphone\n\nPrograms that were not designed for Rhasspy can be used with [adapters](adapters.md).\nFor example, add the following to your `configuration.yaml` (in the `config` directory):\n\n```yaml\nprograms:\n mic:\n arecord:\n command: |\n arecord -q -r 16000 -c 1 -f S16_LE -t raw -\n adapter: |\n mic_adapter_raw.py --rate 16000 --width 2 --channels 1\n\npipelines:\n default:\n mic:\n name: arecord\n```\n\nNow you can run a microphone test:\n\n```sh\nscript/run bin/mic_test_energy.py\n```\n\nWhen speaking, you should see the bar change with volume. If not, check the available devices with `arecord -L` and update the `arecord` command in `configuration.yaml` with `-D ` (prefer devices that start with `plughw:`).\n\nPress CTRL+C to quit.\n\nPipelines will be discussed later. For now, know that the pipeline named `default` will be run if you don't specify one. The mic test script can do this:\n\n```sh\nscript/run bin/mic_test_energy.py --pipeline my-pipeline\n```\n\nYou can also override the mic program:\n\n```sh\nscript/run bin/mic_test_energy.py --mic-program other-program-from-config\n```\n\n\n## Voice Activity Detection\n\nLet's install our first program, [Silero VAD](https://github.com/snakers4/silero-vad/).\nStart by copying from `programs/` to `config/programs`, then run the setup script:\n\n```sh\nmkdir -p config/programs/vad/\ncp -R programs/vad/silero config/programs/vad/\nconfig/programs/vad/silero/script/setup\n```\n\nOnce the setup script completes, add the following to your `configuration.yaml`:\n\n```yaml\nprograms:\n mic: ...\n vad:\n silero:\n command: |\n script/speech_prob \"share/silero_vad.onnx\"\n adapter: |\n vad_adapter_raw.py --rate 16000 --width 2 --channels 1 --samples-per-chunk 512\n\npipelines:\n default:\n mic: ...\n vad:\n name: silero\n```\n\n\nThis calls a command inside `config/programs/vad/silero` and uses an adapter. Notice that the command's working directory will always be `config/programs//`.\n\nYou can test out the voice activity detection (VAD) by recording an audio sample:\n\n```sh\nscript/run bin/mic_record_sample.py sample.wav\n```\n\nSay something for a few seconds and then wait for the program to finish. Afterwards, listen to `sample.wav` and verify that it sounds correct. You may need to adjust microphone settings with `alsamixer`\n\n\n## Speech to Text\n\nNow for the fun part! We'll be installing [faster-whisper](https://github.com/guillaumekln/faster-whisper/), an optimized version of Open AI's [Whisper](https://github.com/openai/whisper) model.\n\n\n```sh\nmkdir -p config/programs/asr/\ncp -R programs/asr/faster-whisper config/programs/asr/\nconfig/programs/asr/faster-whisper/script/setup\n```\n\nBefore using faster-whisper, we need to download a model:\n\n```sh\nconfig/programs/asr/faster-whisper/script/download.py tiny-int8\n```\n\nNotice that the model was downloaded to `config/data/asr/faster-whisper`:\n\n```sh\nfind config/data/asr/faster-whisper/\nconfig/data/asr/faster-whisper/\nconfig/data/asr/faster-whisper/tiny-int8\nconfig/data/asr/faster-whisper/tiny-int8/vocabulary.txt\nconfig/data/asr/faster-whisper/tiny-int8/model.bin\nconfig/data/asr/faster-whisper/tiny-int8/config.json\n```\n\nThe `tiny-int8` model is the smallest and fastest model, but may not give the best transcriptions. Run `download.py` without any arguments to see the available models, or follow [the instructions](https://github.com/guillaumekln/faster-whisper/#model-conversion) to make your own!\n\nAdd the following to `configuration.yaml`:\n\n```yaml\nprograms:\n mic: ...\n vad: ...\n asr:\n faster-whisper:\n command: |\n script/wav2text \"${data_dir}/tiny-int8\" \"{wav_file}\"\n adapter: |\n asr_adapter_wav2text.py\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr:\n name: faster-whisper\n```\n\nYou can now transcribe a voice command:\n\n```sh\nscript/run bin/asr_transcribe.py\n```\n\n(say something)\n\nYou should see a transcription of what you said as part of an [event](wyoming.md).\n\n### Client/Server\n\nSpeech to text systems can take a while to load their models, so a lot of time is wasted if we start from scratch each time.\n\nSome speech to text and text to speech programs have included servers. These usually use [Unix domain sockets](https://en.wikipedia.org/wiki/Unix_domain_socket) to communicate with a small client program.\n\nAdd the following to your `configuration.yaml`:\n\n\n```yaml\nprograms:\n mic: ...\n vad: ...\n asr:\n faster-whisper: ...\n faster-whisper.client:\n command: |\n client_unix_socket.py var/run/faster-whisper.socket\n\nservers:\n asr:\n faster-whisper:\n command: |\n script/server --language \"en\" \"${data_dir}/tiny-int8\"\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr:\n name: faster-whisper.client\n```\n\nStart the server in a separate terminal:\n\n```sh\nscript/run bin/server_run.py asr faster-whisper\n```\n\nWhen it prints \"Ready\", transcribe yourself speaking again:\n\n```sh\nscript/run bin/asr_transcribe.py\n```\n\n(say something)\n\nYou should receive your transcription a bit faster than before.\n\n\n### HTTP Server\n\nRhasspy includes a small HTTP server that allows you to access programs and pipelines over a web API. To get started, run the setup script:\n\n```sh\nscript/setup_http_server\n```\n\nRun HTTP server in a separate terminal:\n\n```sh\nscript/http_server --debug\n```\n\nNow you can transcribe a WAV file over HTTP:\n\n```sh\ncurl -X POST -H 'Content-Type: audio/wav' --data-binary @etc/what_time_is_it.wav 'localhost:13331/asr/transcribe'\n```\n\nYou can run one or more program servers along with the HTTP server:\n\n```sh\nscript/http_server --debug --server asr faster-whisper\n```\n\n**NOTE:** You will need to restart the HTTP server when you change `configuration.yaml`\n\n\n## Wake Word Detection\n\nNext, we'll install [Porcupine](https://github.com/Picovoice/porcupine):\n\n```sh\nmkdir -p config/programs/wake/\ncp -R programs/wake/porcupine1 config/programs/wake/\nconfig/programs/wake/porcupine1/script/setup\n```\n\nCheck available wake word models with:\n\n```sh\nconfig/programs/wake/porcupine1/script/list_models\nalexa_linux.ppn\namericano_linux.ppn\nblueberry_linux.ppn\nbumblebee_linux.ppn\ncomputer_linux.ppn\ngrapefruit_linux.ppn\ngrasshopper_linux.ppn\nhey google_linux.ppn\nhey siri_linux.ppn\njarvis_linux.ppn\nok google_linux.ppn\npico clock_linux.ppn\npicovoice_linux.ppn\nporcupine_linux.ppn\nsmart mirror_linux.ppn\nsnowboy_linux.ppn\nterminator_linux.ppn\nview glass_linux.ppn\n```\n\n**NOTE:** These will be slightly different on a Raspberry Pi (`_raspberry-pi.ppn` instead of `_linux.ppn`).\n\nAdd to `configuration.yaml`:\n\n```yaml\nprograms:\n mic: ...\n vad: ...\n asr: ...\n wake:\n porcupine1:\n command: |\n .venv/bin/python3 bin/porcupine_stream.py --model \"${model}\"\n template_args:\n model: \"porcupine_linux.ppn\"\n\nservers:\n asr: ...\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr: ...\n wake:\n name: porcupine1\n```\n\nNotice that we include `template_args` in the `programs` section. This lets us change specific settings in `pipelines`, which will be demonstrated in a moment.\n\nTest wake word detection:\n\n```sh\nscript/run bin/wake_detect.py --debug\n```\n\n(say \"porcupine\")\n\nNow change the model in `configuration.yaml`:\n\n```yaml\nprograms:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n\nservers:\n asr: ...\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr: ...\n wake:\n name: porcupine1\n template_args:\n model: \"grasshopper_linux.ppn\"\n```\n\nTest wake word detection again:\n\n```sh\nscript/run bin/wake_detect.py --debug\n```\n\n(say \"grasshopper\")\n\nFor non-English models, first download the extra data files:\n\n```sh\nconfig/programs/wake/porcupine1/script/download.py\n```\n\nNext, adjust your `configuration.yaml`. For example, this uses the German keyword \"ananas\":\n\n```yaml\nprograms:\n wake:\n porcupine1:\n command: |\n .venv/bin/python3 bin/porcupine_stream.py --model \"${model}\" --lang_model \"${lang_model}\"\n template_args:\n model: \"${data_dir}/resources/keyword_files_de/linux/ananas_linux.ppn\"\n lang_model: \"${data_dir}/lib/common/porcupine_params_de.pv\"\n\n```\n\nInspect the files in `config/data/wake/porcupine1` for supported languages and keywords. At this time, English, German (de), French (fr), and Spanish (es) are available with keywords for `linux`, `raspberry-pi`, and many other platforms.\n\nGoing back to \"grasshopper\", we can test over HTTP server (restart server):\n\n```sh\ncurl -X POST 'localhost:13331/pipeline/run?stop_after=wake'\n```\n\n(say \"grasshopper\")\n\nTest full voice command:\n\n```sh\ncurl -X POST 'localhost:13331/pipeline/run?stop_after=asr'\n```\n\n(say \"grasshopper\", *pause*, voice command, *wait*)\n\n\n\n## Intent Handling\n\nThere are two types of intent handlers in Rhasspy, ones that handle transcripts directly (text) and others that handle structured intents (name + entities). For this example, we will be handling text directly from `asr`.\n\nIn `configuration.yaml`:\n\n```yaml\nprograms:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n handle:\n date_time:\n command: |\n bin/date_time.py\n adapter: |\n handle_adapter_text.py\n\nservers:\n asr: ...\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n handle:\n name: date_time\n\n```\n\nInstall date time demo script:\n\n```sh\nmkdir -p config/programs/handle/\ncp -R programs/handle/date_time config/programs/handle/\n```\n\nThis script just looks for the words \"date\" and \"time\" in the text, and responds appropriately.\n\nYou can test it on some text:\n\n```sh\necho 'What time is it?' | script/run bin/handle_text.py --debug\n```\n\nNow let's test it with a full voice command:\n\n```sh\nscript/run bin/pipeline_run.py --debug --stop-after handle\n```\n\n(say \"grasshopper\", *pause*, \"what time is it?\")\n\nIt works too over HTTP (restart server):\n\n```sh\ncurl -X POST 'localhost:13331/pipeline/run?stop_after=handle'\n```\n\n(say \"grasshopper\", *pause*, \"what's the date?\")\n\n\n## Text to Speech and Sound\n\nThe final stages of our pipeline will be text to speech (`tts`) and audio output (`snd`).\n\nInstall [Piper](https://github.com/rhasspy/piper):\n\n```sh\nmkdir -p config/programs/tts/\ncp -R programs/tts/piper config/programs/tts/\nconfig/programs/tts/piper/script/setup.py\n```\n\nand download an English voice:\n\n```sh\nconfig/programs/tts/piper/script/download.py english\n```\n\nCall `download.py` without any arguments to see available voices.\n\nAdd to `configuration.yaml`:\n\n```yaml\nprograms:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n handle: ...\n tts:\n piper:\n command: |\n bin/piper --model \"${model}\" --output_file -\n adapter: |\n tts_adapter_text2wav.py\n template_args:\n model: \"${data_dir}/en-us-blizzard_lessac-medium.onnx\"\n snd:\n aplay:\n command: |\n aplay -q -r 22050 -f S16_LE -c 1 -t raw\n adapter: |\n snd_adapter_raw.py --rate 22050 --width 2 --channels 1\n\nservers:\n asr: ...\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n handle: ...\n tts:\n name: piper\n snd:\n name: aplay\n```\n\n\nWe can test the text to speech and audio output programs:\n\n```sh\nscript/run bin/tts_speak.py 'Welcome to the world of speech synthesis.'\n```\n\nThe `bin/tts_synthesize.py` can be used if you want to just output a WAV file.\n\n```sh\nscript/run bin/tts_synthesize.py 'Welcome to the world of speech synthesis.' > welcome.wav\n```\n\nThis also works over HTTP (restart server):\n\n```sh\ncurl -X POST \\\n --data 'Welcome to the world of speech synthesis.' \\\n --output welcome.wav \\\n 'localhost:13331/tts/synthesize'\n```\n\nOr to speak over HTTP:\n\n```sh\ncurl -X POST --data 'Welcome to the world of speech synthesis.' 'localhost:13331/tts/speak'\n```\n\n\n### Client/Server\n\nLike speech to text, text to speech models can take a while to load. Let's add a server for Piper to `configuration.yaml`:\n\n```yaml\nprograms:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n handle: ...\n tts:\n piper.client:\n command: |\n client_unix_socket.py var/run/piper.socket\n snd: ...\n\nservers:\n asr: ...\n tts:\n piper:\n command: |\n script/server \"${model}\"\n template_args:\n model: \"${data_dir}/en-us-blizzard_lessac-medium.onnx\"\n\npipelines:\n default:\n mic: ...\n vad: ...\n asr: ...\n wake: ...\n handle: ...\n tts:\n name: piper.client\n snd: ...\n```\n\nNow we can run both servers with the HTTP server:\n\n```sh\nscript/http_server --debug --server asr faster-whisper --server tts piper\n```\n\nText to speech requests should be faster now.\n\n\n## Complete Pipeline\n\nAs a final example, let's run a complete pipeline from wake word detection to text to speech response:\n\n```sh\nscript/run bin/pipeline_run.py --debug\n```\n\n(say \"grasshopper\", *pause*, \"what time is it?\", *wait*)\n\nRhasspy should speak the current time.\n\nThis also works over HTTP:\n\n```sh\ncurl -X POST 'localhost:13331/pipeline/run'\n```\n\n(say \"grasshopper\", *pause*, \"what is the date?\", *wait*)\n\nRhasspy should speak the current date.\n\n\n## Next Steps\n\n* Connect Rhasspy to [Home Assistant](home_assistant.md)\n* Run one or more [satellites](satellite.md)\n" }, { "path": "docs/wyoming.md", "content": "# The Wyoming Protocol\n\nAn interprocess event protocol over stdin/stdout for Rhasspy v3.\n\n(effectively [JSONL](https://jsonlines.org/) with an optional binary payload)\n\n![Wyoming protocol](img/wyoming.png)\n\n\n## Motivation\n\nRhasspy v2 was built on top of MQTT, and therefore required (1) an MQTT broker and (2) all services to talk over MQTT. Each open source voice program needed a custom service wrapper to talk to Rhasspy.\n\nFor v3, a project goal was to minimize the barrier for programs to talk to Rhasspy.\n\n\n## Talking Directly to Programs\n\nMany voice programs have similar command line interfaces. For example, most text to speech programs accept text through standard input and write a WAV file to standard output or a file:\n\n```sh\necho “Input text” | text-to-speech > output.wav\n```\n\nA protocol based on standard input/output would be universal across languages, operating systems, etc. However, some voice programs need to consume or produce audio/event streams. For example, a speech to text system may return a result much quicker if it can process audio as it's being recorded.\n\n## Event Streams\n\nStandard input/output are byte streams, but they can be easily adapted to event streams that can also carry binary data. This lets us send, for example, chunks of audio to a speech to text program as well as an event to say the stream is finished. All without a broker or a socket!\n\nEach **event** in the Wyoming protocol is:\n\n1. A **single line** of JSON with an object:\n * **MUST** have a `type` field with an event type name\n * MAY have a `data` field with an object that contains event-specific data\n * MAY have a `payload_length` field with a number > 0\n2. If `payload_length` is given, *exactly* that may bytes follows\n\nExample:\n\n```json\n{ \"type\": \"audio-chunk\", \"data\": { \"rate\": 16000, \"width\", \"channels\": 1 }, \"payload_length\": 2048 }\n<2048 bytes>\n```\n\n\n## Adapter\n\nUsing events over standard input/output unfortunately means we cannot talk to most programs directly. Fortunately, [small adapters](adapters.md) can be written and shared for programs with similar command-line interfaces. The adapter speaks events to Rhasspy, but calls the underlying program according to a common convention like “text in, WAV out”.\n\n![Wyoming protocol adapter](img/adapter.png)\n\n## Events Types\n\nVoice programs vary significantly in their options, but programs within the same [domain](domains.md) have the same minimal requirements to function:\n\n* mic\n * Audio input\n * Outputs fixed-sized chunks of PCM audio from a microphone, socket, etc.\n * Audio chunks may contain timestamps\n* wake\n * Wake word detection\n * Inputs fixed-sized chunks of PCM audio\n * Outputs name of detected model, timestamp of audio chunk\n* asr\n * Speech to text\n * Inputs fixed-sized chunks of PCM audio\n * Inputs an event indicating the end of the audio stream (or voice command)\n * Outputs a transcription\n* vad\n * Voice activity detection\n * Inputs fixed-sized chunks of PCM audio\n * Outputs events indicating the beginning and end of a voice command\n* intent\n * Intent recognition\n * Inputs text\n * Outputs an intent with a name and entities (slots)\n* handle\n * Intent/text handling\n * Does something with an intent or directly with a transcription\n * Outputs a text response\n* tts\n * Text to speech\n * Inputs text\n * Outputs one or more fixed-sized chunks of PCM audio\n* snd\n * Audio output\n * Inputs fixed-sized chunks of PCM audio\n * Plays audio through a sound system\n\nThe following event types are currently defined:\n\n| Domain | Type | Data | Payload |\n|--------|----------------|----------------------------------|---------|\n| audio | audio-start | timestamp, rate, width, channels | |\n| audio | audio-chunk | timestamp, rate, width, channels | PCM |\n| audio | audio-stop | timestamp | |\n| wake | detection | name, timestamp | |\n| wake | not-detected | | |\n| vad | voice-started | timestamp | |\n| vad | voice-stopped | timestamp | |\n| asr | transcript | text | |\n| intent | recognize | text | |\n| intent | intent | name, entities | |\n| intent | not-recognized | text | |\n| handle | handled | text | |\n| handle | not-handled | text | |\n| tts | synthesize | text | |\n| snd | played | | |\n" }, { "path": "examples/satellite/configuration.yaml", "content": "satellites:\n default:\n mic:\n name: arecord\n template_args:\n device: \"default\"\n wake:\n name: porcupine1\n template_args:\n model: \"porcupine_raspberry-pi.ppn\"\n remote:\n name: websocket\n template_args:\n uri: \"ws://homeassistant.local:13331/pipeline/asr-tts\"\n snd:\n name: aplay\n template_args:\n device: \"default\"\n" }, { "path": "mypy.ini", "content": "[mypy]\nignore_missing_imports = true\n\n[mypy-setuptools.*]\nignore_missing_imports = True\n" }, { "path": "programs/asr/coqui-stt/README.md", "content": "# Coqui STT\n\nSpeech to text service for Rhasspy based on [Coqui STT](https://stt.readthedocs.io/en/latest/).\n\nAdditional models can be downloaded here: https://coqui.ai/models/\n\n\n## Installation\n\n1. Copy the contents of this directory to `config/programs/asr/coqui-stt/`\n2. Run `script/setup`\n3. Download a model with `script/download.py`\n * Example: `script/download.py en_large`\n * Models are downloaded to `config/data/asr/coqui-stt` directory\n4. Test with `script/wav2text`\n * Example `script/wav2text /path/to/english_v1.0.0-large-vocab/ /path/to/test.wav`\n" }, { "path": "programs/asr/coqui-stt/bin/coqui_stt_raw2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport sys\nfrom pathlib import Path\n\nimport numpy as np\nfrom stt import Model\n\n_LOGGER = logging.getLogger(\"coqui_stt_raw2text\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Coqui STT model directory\")\n parser.add_argument(\n \"--scorer\", help=\"Path to scorer (default: .scorer file in model directory)\"\n )\n parser.add_argument(\n \"--alpha-beta\",\n type=float,\n nargs=2,\n metavar=(\"alpha\", \"beta\"),\n help=\"Scorer alpha/beta\",\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=1024,\n help=\"Number of samples to process at a time\",\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n model_dir = Path(args.model)\n model_path = next(model_dir.glob(\"*.tflite\"))\n if args.scorer:\n scorer_path = Path(args.scorer)\n else:\n scorer_path = next(model_dir.glob(\"*.scorer\"))\n\n _LOGGER.debug(\"Loading model: %s, scorer: %s\", model_path, scorer_path)\n model = Model(str(model_path))\n model.enableExternalScorer(str(scorer_path))\n\n if args.alpha_beta is not None:\n model.setScorerAlphaBeta(*args.alpha_beta)\n\n model_stream = model.createStream()\n chunk = sys.stdin.buffer.read(args.samples_per_chunk)\n _LOGGER.debug(\"Processing audio\")\n while chunk:\n chunk_array = np.frombuffer(chunk, dtype=np.int16)\n model_stream.feedAudioContent(chunk_array)\n chunk = sys.stdin.buffer.read(args.samples_per_chunk)\n\n text = model_stream.finishStream()\n _LOGGER.debug(text)\n\n print(text.strip())\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/coqui-stt/bin/coqui_stt_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport os\nimport socket\nimport threading\nfrom pathlib import Path\n\nimport numpy as np\nfrom stt import Model\n\n_LOGGER = logging.getLogger(\"coqui_stt_server\")\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Coqui STT model directory\")\n parser.add_argument(\n \"--scorer\", help=\"Path to scorer (default: .scorer file in model directory)\"\n )\n parser.add_argument(\n \"--alpha-beta\",\n type=float,\n nargs=2,\n metavar=(\"alpha\", \"beta\"),\n help=\"Scorer alpha/beta\",\n )\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\n \"-r\",\n \"--rate\",\n type=int,\n default=16000,\n help=\"Input audio sample rate (default: 16000)\",\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n model_dir = Path(args.model)\n model_path = next(model_dir.glob(\"*.tflite\"))\n if args.scorer:\n scorer_path = Path(args.scorer)\n else:\n scorer_path = next(model_dir.glob(\"*.scorer\"))\n\n _LOGGER.debug(\"Loading model: %s, scorer: %s\", model_path, scorer_path)\n model = Model(str(model_path))\n model.enableExternalScorer(str(scorer_path))\n\n if args.alpha_beta is not None:\n model.setScorerAlphaBeta(*args.alpha_beta)\n\n _LOGGER.info(\"Ready\")\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n\n # Start new thread for client\n threading.Thread(\n target=handle_client,\n args=(connection, model, args.rate),\n daemon=True,\n ).start()\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\ndef handle_client(connection: socket.socket, model: Model, rate: int) -> None:\n try:\n model_stream = model.createStream()\n is_first_audio = True\n\n with connection, connection.makefile(mode=\"rwb\") as conn_file:\n while True:\n event_info = json.loads(conn_file.readline())\n event_type = event_info[\"type\"]\n\n if event_type == \"audio-chunk\":\n if is_first_audio:\n _LOGGER.debug(\"Receiving audio\")\n is_first_audio = False\n\n num_bytes = event_info[\"payload_length\"]\n chunk = conn_file.read(num_bytes)\n chunk_array = np.frombuffer(chunk, dtype=np.int16)\n model_stream.feedAudioContent(chunk_array)\n elif event_type == \"audio-stop\":\n _LOGGER.info(\"Audio stopped\")\n\n text = model_stream.finishStream()\n transcript_str = (\n json.dumps(\n {\"type\": \"transcript\", \"data\": {\"text\": text}},\n ensure_ascii=False,\n )\n + \"\\n\"\n )\n conn_file.write(transcript_str.encode())\n break\n except Exception:\n _LOGGER.exception(\"Unexpected error in client thread\")\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/coqui-stt/bin/coqui_stt_wav2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport wave\nfrom pathlib import Path\n\nimport numpy as np\nfrom stt import Model\n\n_LOGGER = logging.getLogger(\"coqui_stt_wav2text\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Coqui STT model directory\")\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path to WAV file(s) to transcribe\")\n parser.add_argument(\n \"--scorer\", help=\"Path to scorer (default: .scorer file in model directory)\"\n )\n parser.add_argument(\n \"--alpha-beta\",\n type=float,\n nargs=2,\n metavar=(\"alpha\", \"beta\"),\n help=\"Scorer alpha/beta\",\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n model_dir = Path(args.model)\n model_path = next(model_dir.glob(\"*.tflite\"))\n if args.scorer:\n scorer_path = Path(args.scorer)\n else:\n scorer_path = next(model_dir.glob(\"*.scorer\"))\n\n _LOGGER.debug(\"Loading model: %s, scorer: %s\", model_path, scorer_path)\n model = Model(str(model_path))\n model.enableExternalScorer(str(scorer_path))\n\n if args.alpha_beta is not None:\n model.setScorerAlphaBeta(*args.alpha_beta)\n\n for wav_path in args.wav_file:\n _LOGGER.debug(\"Processing %s\", wav_path)\n wav_file: wave.Wave_read = wave.open(wav_path, \"rb\")\n with wav_file:\n assert wav_file.getframerate() == 16000, \"16Khz sample rate required\"\n assert wav_file.getsampwidth() == 2, \"16-bit samples required\"\n assert wav_file.getnchannels() == 1, \"Mono audio required\"\n audio_bytes = wav_file.readframes(wav_file.getnframes())\n\n model_stream = model.createStream()\n audio_array = np.frombuffer(audio_bytes, dtype=np.int16)\n model_stream.feedAudioContent(audio_array)\n\n text = model_stream.finishStream()\n print(text.strip())\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/coqui-stt/requirements.txt", "content": "stt>=1.4.0,<2.0\nnumpy\n" }, { "path": "programs/asr/coqui-stt/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport itertools\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\nMODELS = {\"en_large\": \"english_v1.0.0-large-vocab\"}\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"model\",\n nargs=\"+\",\n choices=list(itertools.chain(MODELS.keys(), MODELS.values())),\n help=\"Coqui STT model(s) to download\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: share)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/asr_coqui-stt-{model}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if args.destination:\n args.destination = Path(args.destination)\n else:\n # Assume we're in programs/asr/coqui-stt/script\n data_dir = _DIR.parent.parent.parent.parent / \"data\"\n args.destination = data_dir / \"asr\" / \"coqui-stt\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n for model in args.model:\n model = MODELS.get(model, model)\n url = args.link_format.format(model=model)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/coqui-stt/script/raw2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/coqui_stt_raw2text.py\" \"$@\"\n" }, { "path": "programs/asr/coqui-stt/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\npython3 \"${base_dir}/bin/coqui_stt_server.py\" --socketfile \"${socket_dir}/coqui-stt.socket\" \"$@\"\n" }, { "path": "programs/asr/coqui-stt/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/asr/coqui-stt/script/wav2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/coqui_stt_wav2text.py\" \"$@\"\n" }, { "path": "programs/asr/faster-whisper/README.md", "content": "# Faster Whisper\n\nSpeech to text service for Rhasspy based on [faster-whisper](https://github.com/guillaumekln/faster-whisper/).\n\nAdditional models can be downloaded here: https://github.com/rhasspy/models/releases/tag/v1.0\n\n## Installation\n\n1. Copy the contents of this directory to `config/programs/asr/faster-whisper/`\n2. Run `script/setup.py`\n3. Download a model with `script/download.py`\n * Example: `script/download.py tiny-int8`\n * Models are downloaded to `config/data/asr/faster-whisper` directory\n4. Test with `script/wav2text`\n * Example `script/wav2text /path/to/tiny-int8/ /path/to/test.wav`\n" }, { "path": "programs/asr/faster-whisper/bin/faster_whisper_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport io\nimport logging\nimport os\nimport socket\nimport wave\nfrom pathlib import Path\n\nfrom faster_whisper import WhisperModel\n\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.audio import AudioChunk, AudioStop\nfrom rhasspy3.event import read_event, write_event\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to faster-whisper model directory\")\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\n \"--device\",\n default=\"cpu\",\n help=\"Device to use for inference (default: cpu)\",\n )\n parser.add_argument(\n \"--language\",\n help=\"Language to set for transcription\",\n )\n parser.add_argument(\n \"--compute-type\",\n default=\"default\",\n help=\"Compute type (float16, int8, etc.)\",\n )\n parser.add_argument(\n \"--beam-size\",\n type=int,\n default=1,\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n # Load converted faster-whisper model\n model = WhisperModel(\n args.model, device=args.device, compute_type=args.compute_type\n )\n _LOGGER.info(\"Ready\")\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n\n is_first_audio = True\n with connection, connection.makefile(\n mode=\"rwb\"\n ) as conn_file, io.BytesIO() as wav_io:\n wav_file: wave.Wave_write = wave.open(wav_io, \"wb\")\n with wav_file:\n while True:\n event = read_event(conn_file) # type: ignore\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n\n if is_first_audio:\n _LOGGER.debug(\"Receiving audio\")\n wav_file.setframerate(chunk.rate)\n wav_file.setsampwidth(chunk.width)\n wav_file.setnchannels(chunk.channels)\n is_first_audio = False\n\n wav_file.writeframes(chunk.audio)\n elif AudioStop.is_type(event.type):\n _LOGGER.debug(\"Audio stopped\")\n break\n\n wav_io.seek(0)\n segments, _info = model.transcribe(\n wav_io,\n beam_size=args.beam_size,\n language=args.language,\n )\n text = \" \".join(segment.text for segment in segments)\n _LOGGER.info(text)\n\n write_event(Transcript(text=text).event(), conn_file) # type: ignore\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/faster-whisper/bin/faster_whisper_wav2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport time\nfrom pathlib import Path\n\nfrom faster_whisper import WhisperModel\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to faster-whisper model directory\")\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path to WAV file(s) to transcribe\")\n parser.add_argument(\n \"--device\",\n default=\"cpu\",\n help=\"Device to use for inference (default: cpu)\",\n )\n parser.add_argument(\n \"--language\",\n help=\"Language to set for transcription\",\n )\n parser.add_argument(\n \"--compute-type\",\n default=\"default\",\n help=\"Compute type (float16, int8, etc.)\",\n )\n parser.add_argument(\n \"--beam-size\",\n type=int,\n default=1,\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Load converted faster-whisper model\n _LOGGER.debug(\"Loading model: %s\", args.model)\n model = WhisperModel(args.model, device=args.device, compute_type=args.compute_type)\n _LOGGER.info(\"Model loaded\")\n\n for wav_path in args.wav_file:\n _LOGGER.debug(\"Processing %s\", wav_path)\n start_time = time.monotonic_ns()\n segments, _info = model.transcribe(\n wav_path,\n beam_size=args.beam_size,\n language=args.language,\n )\n text = \" \".join(segment.text for segment in segments)\n end_time = time.monotonic_ns()\n _LOGGER.debug(\n \"Transcribed %s in %s second(s)\", wav_path, (end_time - start_time) / 1e9\n )\n _LOGGER.debug(text)\n\n print(text, flush=True)\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/faster-whisper/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\nMODELS = [\n \"tiny\",\n \"tiny-int8\",\n \"base\",\n \"base-int8\",\n \"small\",\n \"small-int8\",\n]\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"model\",\n nargs=\"+\",\n choices=MODELS,\n help=\"faster-whisper model(s) to download\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: share)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/asr_faster-whisper-{model}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if args.destination:\n args.destination = Path(args.destination)\n else:\n # Assume we're in programs/asr/faster-whisper/script\n data_dir = _DIR.parent.parent.parent.parent / \"data\"\n args.destination = data_dir / \"asr\" / \"faster-whisper\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n for model in args.model:\n url = args.link_format.format(model=model)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/faster-whisper/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\npython3 \"${base_dir}/bin/faster_whisper_server.py\" --socketfile \"${socket_dir}/faster-whisper.socket\" \"$@\"\n" }, { "path": "programs/asr/faster-whisper/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -e \"${base_dir}/src\"\n\n# Install rhasspy3\nrhasspy3_dir=\"${base_dir}/../../../..\"\npip3 install -e \"${rhasspy3_dir}\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/asr/faster-whisper/script/wav2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/faster_whisper_wav2text.py\" \"$@\"\n" }, { "path": "programs/asr/faster-whisper/src/LICENSE", "content": "MIT License\n\nCopyright (c) 2023 Guillaume Klein\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "programs/asr/faster-whisper/src/README.md", "content": "# Faster Whisper transcription with CTranslate2\n\nThis repository demonstrates how to implement the Whisper transcription using [CTranslate2](https://github.com/OpenNMT/CTranslate2/), which is a fast inference engine for Transformer models.\n\nThis implementation is about 4 times faster than [openai/whisper](https://github.com/openai/whisper) for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU.\n\n## Installation\n\n```bash\npip install -e .[conversion]\n```\n\nThe model conversion requires the modules `transformers` and `torch` which are installed by the `[conversion]` requirement. Once a model is converted, these modules are no longer needed and the installation could be simplified to:\n\n```bash\npip install -e .\n```\n\n## Usage\n\n### Model conversion\n\nA Whisper model should be first converted into the CTranslate2 format. For example the command below converts the \"medium\" Whisper model and saves the weights in FP16:\n\n```bash\nct2-transformers-converter --model openai/whisper-medium --output_dir whisper-medium-ct2 --quantization float16\n```\n\nIf needed, models can also be converted from the code. See the [conversion API](https://opennmt.net/CTranslate2/python/ctranslate2.converters.TransformersConverter.html).\n\n### Transcription\n\n```python\nfrom faster_whisper import WhisperModel\n\nmodel_path = \"whisper-medium-ct2/\"\n\n# Run on GPU with FP16\nmodel = WhisperModel(model_path, device=\"cuda\", compute_type=\"float16\")\n\n# or run on GPU with INT8\n# model = WhisperModel(model_path, device=\"cuda\", compute_type=\"int8_float16\")\n# or run on CPU with INT8\n# model = WhisperModel(model_path, device=\"cpu\", compute_type=\"int8\")\n\nsegments, info = model.transcribe(\"audio.mp3\", beam_size=5)\n\nprint(\"Detected language '%s' with probability %f\" % (info.language, info.language_probability))\n\nfor segment in segments:\n print(\"[%ds -> %ds] %s\" % (segment.start, segment.end, segment.text))\n```\n\n## Comparing performance against openai/whisper\n\nIf you are comparing the performance against [openai/whisper](https://github.com/openai/whisper), you should make sure to use the same settings in both frameworks. In particular:\n\n* In openai/whisper, `model.transcribe` uses a beam size of 1 by default. A different beam size will have an important impact on performance so make sure to use the same.\n* When running on CPU, make sure to set the same number of threads. Both frameworks will read the environment variable `OMP_NUM_THREADS`, which can be set when running your script:\n\n```bash\nOMP_NUM_THREADS=4 python3 my_script.py\n```\n" }, { "path": "programs/asr/faster-whisper/src/faster_whisper/__init__.py", "content": "from faster_whisper.transcribe import WhisperModel\n" }, { "path": "programs/asr/faster-whisper/src/faster_whisper/audio.py", "content": "import av\nimport numpy as np\n\n\ndef decode_audio(input_file, sampling_rate=16000):\n \"\"\"Decodes the audio.\n\n Args:\n input_file: Path to the input file or a file-like object.\n sampling_rate: Resample the audio to this sample rate.\n\n Returns:\n A float32 Numpy array.\n \"\"\"\n fifo = av.audio.fifo.AudioFifo()\n resampler = av.audio.resampler.AudioResampler(\n format=\"s16\",\n layout=\"mono\",\n rate=sampling_rate,\n )\n\n with av.open(input_file) as container:\n # Decode and resample each audio frame.\n for frame in container.decode(audio=0):\n frame.pts = None\n for new_frame in resampler.resample(frame):\n fifo.write(new_frame)\n\n # Flush the resampler.\n for new_frame in resampler.resample(None):\n fifo.write(new_frame)\n\n frame = fifo.read()\n\n # Convert s16 back to f32.\n return frame.to_ndarray().flatten().astype(np.float32) / 32768.0\n" }, { "path": "programs/asr/faster-whisper/src/faster_whisper/feature_extractor.py", "content": "import numpy as np\n\n\n# Adapted from https://github.com/huggingface/transformers/blob/main/src/transformers/models/whisper/feature_extraction_whisper.py\nclass FeatureExtractor:\n def __init__(\n self,\n feature_size=80,\n sampling_rate=16000,\n hop_length=160,\n chunk_length=30,\n n_fft=400,\n ):\n self.n_fft = n_fft\n self.hop_length = hop_length\n self.chunk_length = chunk_length\n self.n_samples = chunk_length * sampling_rate\n self.nb_max_frames = self.n_samples // hop_length\n self.time_per_frame = hop_length / sampling_rate\n self.sampling_rate = sampling_rate\n self.mel_filters = self.get_mel_filters(\n sampling_rate, n_fft, n_mels=feature_size\n )\n\n def get_mel_filters(self, sr, n_fft, n_mels=128, dtype=np.float32):\n # Initialize the weights\n n_mels = int(n_mels)\n weights = np.zeros((n_mels, int(1 + n_fft // 2)), dtype=dtype)\n\n # Center freqs of each FFT bin\n fftfreqs = np.fft.rfftfreq(n=n_fft, d=1.0 / sr)\n\n # 'Center freqs' of mel bands - uniformly spaced between limits\n min_mel = 0.0\n max_mel = 45.245640471924965\n\n mels = np.linspace(min_mel, max_mel, n_mels + 2)\n\n mels = np.asanyarray(mels)\n\n # Fill in the linear scale\n f_min = 0.0\n f_sp = 200.0 / 3\n freqs = f_min + f_sp * mels\n\n # And now the nonlinear scale\n min_log_hz = 1000.0 # beginning of log region (Hz)\n min_log_mel = (min_log_hz - f_min) / f_sp # same (Mels)\n logstep = np.log(6.4) / 27.0 # step size for log region\n\n # If we have vector data, vectorize\n log_t = mels >= min_log_mel\n freqs[log_t] = min_log_hz * np.exp(logstep * (mels[log_t] - min_log_mel))\n\n mel_f = freqs\n\n fdiff = np.diff(mel_f)\n ramps = np.subtract.outer(mel_f, fftfreqs)\n\n for i in range(n_mels):\n # lower and upper slopes for all bins\n lower = -ramps[i] / fdiff[i]\n upper = ramps[i + 2] / fdiff[i + 1]\n\n # .. then intersect them with each other and zero\n weights[i] = np.maximum(0, np.minimum(lower, upper))\n\n # Slaney-style mel is scaled to be approx constant energy per channel\n enorm = 2.0 / (mel_f[2 : n_mels + 2] - mel_f[:n_mels])\n weights *= enorm[:, np.newaxis]\n\n return weights\n\n def fram_wave(self, waveform, center=True):\n \"\"\"\n Transform a raw waveform into a list of smaller waveforms.\n The window length defines how much of the signal is\n contain in each frame (smalle waveform), while the hope length defines the step\n between the beginning of each new frame.\n Centering is done by reflecting the waveform which is first centered around\n `frame_idx * hop_length`.\n \"\"\"\n frames = []\n for i in range(0, waveform.shape[0] + 1, self.hop_length):\n half_window = (self.n_fft - 1) // 2 + 1\n if center:\n start = i - half_window if i > half_window else 0\n end = (\n i + half_window\n if i < waveform.shape[0] - half_window\n else waveform.shape[0]\n )\n\n frame = waveform[start:end]\n\n if start == 0:\n padd_width = (-i + half_window, 0)\n frame = np.pad(frame, pad_width=padd_width, mode=\"reflect\")\n\n elif end == waveform.shape[0]:\n padd_width = (0, (i - waveform.shape[0] + half_window))\n frame = np.pad(frame, pad_width=padd_width, mode=\"reflect\")\n\n else:\n frame = waveform[i : i + self.n_fft]\n frame_width = frame.shape[0]\n if frame_width < waveform.shape[0]:\n frame = np.lib.pad(\n frame,\n pad_width=(0, self.n_fft - frame_width),\n mode=\"constant\",\n constant_values=0,\n )\n\n frames.append(frame)\n return np.stack(frames, 0)\n\n def stft(self, frames, window):\n \"\"\"\n Calculates the complex Short-Time Fourier Transform (STFT) of the given framed signal.\n Should give the same results as `torch.stft`.\n \"\"\"\n frame_size = frames.shape[1]\n fft_size = self.n_fft\n\n if fft_size is None:\n fft_size = frame_size\n\n if fft_size < frame_size:\n raise ValueError(\"FFT size must greater or equal the frame size\")\n # number of FFT bins to store\n num_fft_bins = (fft_size >> 1) + 1\n\n data = np.empty((len(frames), num_fft_bins), dtype=np.complex64)\n fft_signal = np.zeros(fft_size)\n\n for f, frame in enumerate(frames):\n if window is not None:\n np.multiply(frame, window, out=fft_signal[:frame_size])\n else:\n fft_signal[:frame_size] = frame\n data[f] = np.fft.fft(fft_signal, axis=0)[:num_fft_bins]\n return data.T\n\n def __call__(self, waveform):\n \"\"\"\n Compute the log-Mel spectrogram of the provided audio, gives similar results\n whisper's original torch implementation with 1e-5 tolerance.\n \"\"\"\n window = np.hanning(self.n_fft + 1)[:-1]\n\n frames = self.fram_wave(waveform)\n stft = self.stft(frames, window=window)\n magnitudes = np.abs(stft[:, :-1]) ** 2\n\n filters = self.mel_filters\n mel_spec = filters @ magnitudes\n\n log_spec = np.log10(np.clip(mel_spec, a_min=1e-10, a_max=None))\n log_spec = np.maximum(log_spec, log_spec.max() - 8.0)\n log_spec = (log_spec + 4.0) / 4.0\n\n return log_spec\n" }, { "path": "programs/asr/faster-whisper/src/faster_whisper/transcribe.py", "content": "import collections\nimport os\nimport zlib\n\nimport ctranslate2\nimport numpy as np\nimport tokenizers\nfrom faster_whisper.audio import decode_audio\nfrom faster_whisper.feature_extractor import FeatureExtractor\n\n\nclass Segment(collections.namedtuple(\"Segment\", (\"start\", \"end\", \"text\"))):\n pass\n\n\nclass AudioInfo(\n collections.namedtuple(\"AudioInfo\", (\"language\", \"language_probability\"))\n):\n pass\n\n\nclass TranscriptionOptions(\n collections.namedtuple(\n \"TranscriptionOptions\",\n (\n \"beam_size\",\n \"best_of\",\n \"patience\",\n \"log_prob_threshold\",\n \"no_speech_threshold\",\n \"compression_ratio_threshold\",\n \"condition_on_previous_text\",\n \"temperatures\",\n ),\n )\n):\n pass\n\n\nclass WhisperModel:\n def __init__(\n self,\n model_path,\n device=\"auto\",\n compute_type=\"default\",\n cpu_threads=0,\n ):\n \"\"\"Initializes the Whisper model.\n\n Args:\n model_path: Path to the converted model.\n device: Device to use for computation (\"cpu\", \"cuda\", \"auto\").\n compute_type: Type to use for computation.\n See https://opennmt.net/CTranslate2/quantization.html.\n cpu_threads: Number of threads to use when running on CPU (4 by default).\n A non zero value overrides the OMP_NUM_THREADS environment variable.\n \"\"\"\n self.model = ctranslate2.models.Whisper(\n model_path,\n device=device,\n compute_type=compute_type,\n intra_threads=cpu_threads,\n )\n\n self.feature_extractor = FeatureExtractor()\n self.decoder = tokenizers.decoders.ByteLevel()\n\n with open(os.path.join(model_path, \"vocabulary.txt\")) as vocab_file:\n self.ids_to_tokens = [line.rstrip(\"\\n\") for line in vocab_file]\n self.tokens_to_ids = {\n token: i for i, token in enumerate(self.ids_to_tokens)\n }\n\n self.eot_id = self.tokens_to_ids[\"<|endoftext|>\"]\n self.timestamp_begin_id = self.tokens_to_ids[\"<|notimestamps|>\"] + 1\n self.input_stride = 2\n self.time_precision = 0.02\n self.max_length = 448\n\n def transcribe(\n self,\n input_file,\n language=None,\n beam_size=5,\n best_of=5,\n patience=1,\n temperature=[0.0, 0.2, 0.4, 0.6, 0.8, 1.0],\n compression_ratio_threshold=2.4,\n log_prob_threshold=-1.0,\n no_speech_threshold=0.6,\n condition_on_previous_text=True,\n ):\n \"\"\"Transcribes an input file.\n\n Arguments:\n input_file: Path to the input file or a file-like object.\n language: The language spoken in the audio. If not set, the language will be\n detected in the first 30 seconds of audio.\n beam_size: Beam size to use for decoding.\n best_of: Number of candidates when sampling with non-zero temperature.\n patience: Beam search patience factor.\n temperature: Temperature for sampling. It can be a tuple of temperatures,\n which will be successively used upon failures according to either\n `compression_ratio_threshold` or `logprob_threshold`.\n compression_ratio_threshold: If the gzip compression ratio is above this value,\n treat as failed.\n log_prob_threshold: If the average log probability over sampled tokens is\n below this value, treat as failed.\n no_speech_threshold: If the no_speech probability is higher than this value AND\n the average log probability over sampled tokens is below `logprob_threshold`,\n consider the segment as silent.\n condition_on_previous_text: If True, the previous output of the model is provided\n as a prompt for the next window; disabling may make the text inconsistent across\n windows, but the model becomes less prone to getting stuck in a failure loop,\n such as repetition looping or timestamps going out of sync.\n\n Returns:\n A tuple with:\n\n - a generator over transcribed segments\n - an instance of AudioInfo\n \"\"\"\n audio = decode_audio(\n input_file, sampling_rate=self.feature_extractor.sampling_rate\n )\n features = self.feature_extractor(audio)\n\n if language is None:\n segment = self.get_segment(features)\n input = self.get_input(segment)\n results = self.model.detect_language(input)\n language_token, language_probability = results[0][0]\n language = language_token[2:-2]\n else:\n language_probability = 1\n\n options = TranscriptionOptions(\n beam_size=beam_size,\n best_of=best_of,\n patience=patience,\n log_prob_threshold=log_prob_threshold,\n no_speech_threshold=no_speech_threshold,\n compression_ratio_threshold=compression_ratio_threshold,\n condition_on_previous_text=condition_on_previous_text,\n temperatures=(\n temperature if isinstance(temperature, (list, tuple)) else [temperature]\n ),\n )\n\n segments = self.generate_segments(features, language, options)\n\n audio_info = AudioInfo(\n language=language,\n language_probability=language_probability,\n )\n\n return segments, audio_info\n\n def generate_segments(self, features, language, options):\n tokenized_segments = self.generate_tokenized_segments(\n features, language, options\n )\n\n for start, end, tokens in tokenized_segments:\n text = self.decode_text_tokens(tokens)\n if not text.strip():\n continue\n\n yield Segment(\n start=start,\n end=end,\n text=text,\n )\n\n def generate_tokenized_segments(self, features, language, options):\n num_frames = features.shape[-1]\n offset = 0\n all_tokens = []\n prompt_reset_since = 0\n\n while offset < num_frames:\n time_offset = offset * self.feature_extractor.time_per_frame\n segment = self.get_segment(features, offset)\n segment_duration = segment.shape[-1] * self.feature_extractor.time_per_frame\n\n previous_tokens = all_tokens[prompt_reset_since:]\n prompt = self.get_prompt(language, previous_tokens)\n result, temperature = self.generate_with_fallback(segment, prompt, options)\n\n if (\n result.no_speech_prob > options.no_speech_threshold\n and result.scores[0] < options.log_prob_threshold\n ):\n offset += segment.shape[-1]\n continue\n\n tokens = result.sequences_ids[0]\n\n consecutive_timestamps = [\n i\n for i in range(len(tokens))\n if i > 0\n and tokens[i] >= self.timestamp_begin_id\n and tokens[i - 1] >= self.timestamp_begin_id\n ]\n\n if len(consecutive_timestamps) > 0:\n last_slice = 0\n for i, current_slice in enumerate(consecutive_timestamps):\n sliced_tokens = tokens[last_slice:current_slice]\n start_timestamp_position = (\n sliced_tokens[0] - self.timestamp_begin_id\n )\n end_timestamp_position = sliced_tokens[-1] - self.timestamp_begin_id\n start_time = (\n time_offset + start_timestamp_position * self.time_precision\n )\n end_time = (\n time_offset + end_timestamp_position * self.time_precision\n )\n\n last_in_window = i + 1 == len(consecutive_timestamps)\n\n # Include the last timestamp so that all tokens are included in a segment.\n if last_in_window:\n sliced_tokens.append(tokens[current_slice])\n\n yield start_time, end_time, sliced_tokens\n last_slice = current_slice\n\n last_timestamp_position = (\n tokens[last_slice - 1] - self.timestamp_begin_id\n )\n offset += last_timestamp_position * self.input_stride\n all_tokens.extend(tokens[: last_slice + 1])\n\n else:\n duration = segment_duration\n timestamps = [\n token for token in tokens if token >= self.timestamp_begin_id\n ]\n if len(timestamps) > 0 and timestamps[-1] != self.timestamp_begin_id:\n last_timestamp_position = timestamps[-1] - self.timestamp_begin_id\n duration = last_timestamp_position * self.time_precision\n\n yield time_offset, time_offset + duration, tokens\n\n offset += segment.shape[-1]\n all_tokens.extend(tokens)\n\n if not options.condition_on_previous_text or temperature > 0.5:\n prompt_reset_since = len(all_tokens)\n\n def decode_text_tokens(self, tokens):\n text_tokens = [\n self.ids_to_tokens[token] for token in tokens if token < self.eot_id\n ]\n\n return self.decoder.decode(text_tokens)\n\n def generate_with_fallback(self, segment, prompt, options):\n features = self.get_input(segment)\n result = None\n final_temperature = None\n\n for temperature in options.temperatures:\n if temperature > 0:\n kwargs = {\n \"beam_size\": 1,\n \"num_hypotheses\": options.best_of,\n \"sampling_topk\": 0,\n \"sampling_temperature\": temperature,\n }\n else:\n kwargs = {\n \"beam_size\": options.beam_size,\n \"patience\": options.patience,\n }\n\n final_temperature = temperature\n result = self.model.generate(\n features,\n [prompt],\n max_length=self.max_length,\n return_scores=True,\n return_no_speech_prob=True,\n **kwargs,\n )[0]\n\n tokens = result.sequences_ids[0]\n text = self.decode_text_tokens(tokens)\n compression_ratio = get_compression_ratio(text)\n\n if (\n compression_ratio <= options.compression_ratio_threshold\n and result.scores[0] >= options.log_prob_threshold\n ):\n break\n\n return result, final_temperature\n\n def get_prompt(self, language, previous_tokens):\n prompt = []\n\n if previous_tokens:\n prompt.append(self.tokens_to_ids[\"<|startofprev|>\"])\n prompt.extend(previous_tokens[-(self.max_length // 2 - 1) :])\n\n prompt += [\n self.tokens_to_ids[\"<|startoftranscript|>\"],\n self.tokens_to_ids[\"<|%s|>\" % language],\n self.tokens_to_ids[\"<|transcribe|>\"],\n ]\n\n return prompt\n\n def get_segment(self, features, offset=0):\n if offset > 0:\n features = features[:, offset:]\n\n num_frames = features.shape[-1]\n required_num_frames = self.feature_extractor.nb_max_frames\n\n if num_frames > required_num_frames:\n features = features[:, :required_num_frames]\n elif num_frames < required_num_frames:\n pad_widths = [(0, 0), (0, required_num_frames - num_frames)]\n features = np.pad(features, pad_widths)\n\n features = np.ascontiguousarray(features)\n return features\n\n def get_input(self, segment):\n segment = np.expand_dims(segment, 0)\n segment = ctranslate2.StorageView.from_array(segment)\n return segment\n\n\ndef get_compression_ratio(text):\n text_bytes = text.encode(\"utf-8\")\n return len(text_bytes) / len(zlib.compress(text_bytes))\n" }, { "path": "programs/asr/faster-whisper/src/requirements.conversion.txt", "content": "transformers[torch]>=4.23\n" }, { "path": "programs/asr/faster-whisper/src/requirements.txt", "content": "av==10.*\nctranslate2>=3.5,<4\ntokenizers==0.13.*\n" }, { "path": "programs/asr/faster-whisper/src/setup.py", "content": "import os\n\nfrom setuptools import find_packages, setup\n\n\ndef get_requirements(path):\n with open(path, encoding=\"utf-8\") as requirements:\n return [requirement.strip() for requirement in requirements]\n\n\nbase_dir = os.path.dirname(os.path.abspath(__file__))\ninstall_requires = get_requirements(os.path.join(base_dir, \"requirements.txt\"))\nconversion_requires = get_requirements(\n os.path.join(base_dir, \"requirements.conversion.txt\")\n)\n\nsetup(\n name=\"faster-whisper\",\n version=\"0.1.0\",\n description=\"Faster Whisper transcription with CTranslate2\",\n author=\"Guillaume Klein\",\n python_requires=\">=3.7\",\n install_requires=install_requires,\n extras_require={\n \"conversion\": conversion_requires,\n },\n packages=find_packages(),\n)\n" }, { "path": "programs/asr/pocketsphinx/README.md", "content": "# Pocketsphinx\n\nSpeech to text service for Rhasspy based on [Pocketsphinx](https://github.com/cmusphinx/pocketsphinx).\n\nAdditional models can be downloaded here: https://github.com/synesthesiam/voice2json-profiles\n\nModel directories should have this layout:\n\n* model/\n * acoustic_model/\n * dictionary.txt\n * language_model.txt\n \nThese correspond to the `-hmm`, `-dict`, and `-lm` decoder arguments.\n\n## Installation\n\n1. Copy the contents of this directory to `config/programs/asr/pocketsphinx/`\n2. Run `script/setup`\n3. Download a model with `script/download.py`\n * Example: `script/download.py en_cmu`\n * Models are downloaded to `config/data/asr/pocketsphinx` directory\n4. Test with `script/wav2text`\n * Example `script/wav2text /path/to/en-us_pocketsphinx-cmu/ /path/to/test.wav`\n" }, { "path": "programs/asr/pocketsphinx/bin/pocketsphinx_raw2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport sys\nfrom pathlib import Path\n\nimport pocketsphinx\n\n_LOGGER = logging.getLogger(\"pocketsphinx_raw2text\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Pocketsphinx model directory\")\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=1024,\n help=\"Number of samples to process at a time\",\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n model_dir = Path(args.model)\n\n _LOGGER.debug(\"Loading model from %s\", model_dir.absolute())\n decoder_config = pocketsphinx.Decoder.default_config()\n decoder_config.set_string(\"-hmm\", str(model_dir / \"acoustic_model\"))\n decoder_config.set_string(\"-dict\", str(model_dir / \"dictionary.txt\"))\n decoder_config.set_string(\"-lm\", str(model_dir / \"language_model.txt\"))\n decoder = pocketsphinx.Decoder(decoder_config)\n\n decoder.start_utt()\n\n chunk = sys.stdin.buffer.read(args.samples_per_chunk)\n _LOGGER.debug(\"Processing audio\")\n while chunk:\n decoder.process_raw(chunk, False, False)\n chunk = sys.stdin.buffer.read(args.samples_per_chunk)\n\n decoder.end_utt()\n hyp = decoder.hyp()\n if hyp:\n text = hyp.hypstr\n else:\n text = \"\"\n\n _LOGGER.debug(text)\n\n print(text.strip())\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/pocketsphinx/bin/pocketsphinx_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport os\nimport socket\nimport threading\nfrom pathlib import Path\n\nimport pocketsphinx\n\n_LOGGER = logging.getLogger(\"pocketsphinx_server\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Pocketsphinx model directory\")\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\n \"-r\",\n \"--rate\",\n type=int,\n default=16000,\n help=\"Input audio sample rate (default: 16000)\",\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n model_dir = Path(args.model)\n\n decoder_config = pocketsphinx.Decoder.default_config()\n decoder_config.set_string(\"-hmm\", str(model_dir / \"acoustic_model\"))\n decoder_config.set_string(\"-dict\", str(model_dir / \"dictionary.txt\"))\n decoder_config.set_string(\"-lm\", str(model_dir / \"language_model.txt\"))\n decoder = pocketsphinx.Decoder(decoder_config)\n\n _LOGGER.info(\"Ready\")\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n\n # Start new thread for client\n threading.Thread(\n target=handle_client,\n args=(connection, decoder, args.rate),\n daemon=True,\n ).start()\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\ndef handle_client(\n connection: socket.socket, decoder: pocketsphinx.Decoder, rate: int\n) -> None:\n try:\n decoder.start_utt()\n is_first_audio = True\n\n with connection, connection.makefile(mode=\"rwb\") as conn_file:\n while True:\n event_info = json.loads(conn_file.readline())\n event_type = event_info[\"type\"]\n\n if event_type == \"audio-chunk\":\n if is_first_audio:\n _LOGGER.debug(\"Receiving audio\")\n is_first_audio = False\n\n num_bytes = event_info[\"payload_length\"]\n chunk = conn_file.read(num_bytes)\n decoder.process_raw(chunk, False, False)\n elif event_type == \"audio-stop\":\n _LOGGER.info(\"Audio stopped\")\n\n decoder.end_utt()\n hyp = decoder.hyp()\n if hyp:\n text = hyp.hypstr.strip()\n else:\n text = \"\"\n\n transcript_str = (\n json.dumps(\n {\"type\": \"transcript\", \"data\": {\"text\": text}},\n ensure_ascii=False,\n )\n + \"\\n\"\n )\n conn_file.write(transcript_str.encode())\n break\n except Exception:\n _LOGGER.exception(\"Unexpected error in client thread\")\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/pocketsphinx/bin/pocketsphinx_wav2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport wave\nfrom pathlib import Path\n\nimport pocketsphinx\n\n_LOGGER = logging.getLogger(\"pocketsphinx_wav2text\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Pocketsphinx model directory\")\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path to WAV file(s) to transcribe\")\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n model_dir = Path(args.model)\n\n _LOGGER.debug(\"Loading model from %s\", model_dir.absolute())\n decoder_config = pocketsphinx.Decoder.default_config()\n decoder_config.set_string(\"-hmm\", str(model_dir / \"acoustic_model\"))\n decoder_config.set_string(\"-dict\", str(model_dir / \"dictionary.txt\"))\n decoder_config.set_string(\"-lm\", str(model_dir / \"language_model.txt\"))\n decoder = pocketsphinx.Decoder(decoder_config)\n\n for wav_path in args.wav_file:\n _LOGGER.debug(\"Processing %s\", wav_path)\n wav_file: wave.Wave_read = wave.open(wav_path, \"rb\")\n with wav_file:\n assert wav_file.getframerate() == 16000, \"16Khz sample rate required\"\n assert wav_file.getsampwidth() == 2, \"16-bit samples required\"\n assert wav_file.getnchannels() == 1, \"Mono audio required\"\n audio_bytes = wav_file.readframes(wav_file.getnframes())\n\n decoder.start_utt()\n decoder.process_raw(audio_bytes, False, True)\n decoder.end_utt()\n hyp = decoder.hyp()\n if hyp:\n text = hyp.hypstr\n else:\n text = \"\"\n\n print(text.strip())\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/pocketsphinx/requirements.txt", "content": "pocketsphinx @ https://github.com/synesthesiam/pocketsphinx-python/releases/download/v1.0/pocketsphinx-python.tar.gz\n" }, { "path": "programs/asr/pocketsphinx/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport itertools\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\nMODELS = {\"en_cmu\": \"en-us_pocketsphinx-cmu\"}\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"model\",\n nargs=\"+\",\n choices=list(itertools.chain(MODELS.keys(), MODELS.values())),\n help=\"Pocketsphinx model(s) to download\",\n )\n parser.add_argument(\"--destination\", help=\"Path to destination directory\")\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/asr_pocketsphinx-{model}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if args.destination:\n args.destination = Path(args.destination)\n else:\n # Assume we're in programs/asr/pocketsphinx/script\n data_dir = _DIR.parent.parent.parent.parent / \"data\"\n args.destination = data_dir / \"asr\" / \"pocketsphinx\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n for model in args.model:\n model = MODELS.get(model, model)\n url = args.link_format.format(model=model)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/pocketsphinx/script/raw2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/pocketsphinx_raw2text.py\" \"$@\"\n" }, { "path": "programs/asr/pocketsphinx/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\npython3 \"${base_dir}/bin/pocketsphinx_server.py\" --socketfile \"${socket_dir}/pocketsphinx.socket\" \"$@\"\n" }, { "path": "programs/asr/pocketsphinx/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/asr/pocketsphinx/script/wav2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/pocketsphinx_wav2text.py\" \"$@\"\n" }, { "path": "programs/asr/vosk/README.md", "content": "# Vosk\n\nSpeech to text service for Rhasspy based on [Vosk](https://alphacephei.com/vosk/).\n\nYou can download additional models here: https://alphacephei.com/vosk/models\n\n\n## Installation\n\n1. Copy the contents of this directory to `config/programs/asr/vosk/`\n2. Run `script/setup`\n3. Download a model with `script/download.py`\n * Example: `script/download.py en_small`\n * Models are downloaded to `config/data/asr/vosk` directory\n4. Test with `script/wav2text`\n * Example `script/wav2text /path/to/vosk-model-small-en-us-0.15/ /path/to/test.wav`\n" }, { "path": "programs/asr/vosk/bin/vosk_raw2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport sys\n\nfrom vosk import KaldiRecognizer, Model, SetLogLevel\n\n_LOGGER = logging.getLogger(\"vosk_raw2text\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Vosk model directory\")\n parser.add_argument(\n \"-r\",\n \"--rate\",\n type=int,\n default=16000,\n help=\"Model sample rate (default: 16000)\",\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=1024,\n help=\"Number of samples to process at a time\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n SetLogLevel(0)\n\n model = Model(args.model)\n recognizer = KaldiRecognizer(\n model,\n args.rate,\n )\n\n chunk = sys.stdin.buffer.read(args.samples_per_chunk)\n _LOGGER.debug(\"Processing audio\")\n while chunk:\n recognizer.AcceptWaveform(chunk)\n chunk = sys.stdin.buffer.read(args.samples_per_chunk)\n\n result = json.loads(recognizer.FinalResult())\n print(result[\"text\"].strip())\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/vosk/bin/vosk_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport os\nimport socket\nimport threading\n\nfrom vosk import KaldiRecognizer, Model, SetLogLevel\n\n_LOGGER = logging.getLogger(\"vosk_server\")\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Vosk model directory\")\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\n \"-r\",\n \"--rate\",\n type=int,\n default=16000,\n help=\"Input audio sample rate (default: 16000)\",\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n # Load Kaldi model\n SetLogLevel(0)\n model = Model(args.model)\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n\n # Start new thread for client\n threading.Thread(\n target=handle_client,\n args=(connection, model, args.rate),\n daemon=True,\n ).start()\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\ndef handle_client(connection: socket.socket, model: Model, rate: int) -> None:\n try:\n recognizer = KaldiRecognizer(\n model,\n rate,\n )\n is_first_audio = True\n\n with connection, connection.makefile(mode=\"rwb\") as conn_file:\n while True:\n event_info = json.loads(conn_file.readline())\n event_type = event_info[\"type\"]\n\n if event_type == \"audio-chunk\":\n if is_first_audio:\n _LOGGER.debug(\"Receiving audio\")\n is_first_audio = False\n\n num_bytes = event_info[\"payload_length\"]\n chunk = conn_file.read(num_bytes)\n recognizer.AcceptWaveform(chunk)\n elif event_type == \"audio-stop\":\n _LOGGER.info(\"Audio stopped\")\n\n result = json.loads(recognizer.FinalResult())\n _LOGGER.info(result)\n text = result[\"text\"]\n\n transcript_str = (\n json.dumps(\n {\"type\": \"transcript\", \"data\": {\"text\": text}},\n ensure_ascii=False,\n )\n + \"\\n\"\n )\n conn_file.write(transcript_str.encode())\n break\n except Exception:\n _LOGGER.exception(\"Unexpected error in client thread\")\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/vosk/bin/vosk_wav2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport wave\nfrom pathlib import Path\n\nfrom vosk import KaldiRecognizer, Model, SetLogLevel\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Vosk model directory\")\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path to WAV file(s) to transcribe\")\n parser.add_argument(\n \"-r\",\n \"--rate\",\n type=int,\n default=16000,\n help=\"Model sample rate (default: 16000)\",\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=1024,\n help=\"Number of samples to process at a time\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n SetLogLevel(0)\n\n model = Model(args.model)\n recognizer = KaldiRecognizer(\n model,\n args.rate,\n )\n\n for wav_path in args.wav_file:\n _LOGGER.debug(\"Processing %s\", wav_path)\n wav_file: wave.Wave_read = wave.open(wav_path, \"rb\")\n with wav_file:\n assert wav_file.getframerate() == 16000, \"16Khz sample rate required\"\n assert wav_file.getsampwidth() == 2, \"16-bit samples required\"\n assert wav_file.getnchannels() == 1, \"Mono audio required\"\n audio_bytes = wav_file.readframes(wav_file.getnframes())\n recognizer.AcceptWaveform(audio_bytes)\n\n result = json.loads(recognizer.FinalResult())\n print(result[\"text\"].strip())\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/vosk/requirements.txt", "content": "vosk\n" }, { "path": "programs/asr/vosk/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport itertools\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\nMODELS = {\"en_medium\": \"en-us-0.22-lgraph\", \"en_small\": \"small-en-us-0.15\"}\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"model\",\n nargs=\"+\",\n choices=list(itertools.chain(MODELS.keys(), MODELS.values())),\n help=\"Vosk model(s) to download\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: share)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/asr_vosk-model-{model}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if args.destination:\n args.destination = Path(args.destination)\n else:\n # Assume we're in programs/asr/vosk/script\n data_dir = _DIR.parent.parent.parent.parent / \"data\"\n args.destination = data_dir / \"asr\" / \"vosk\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n for model in args.model:\n model = MODELS.get(model, model)\n url = args.link_format.format(model=model)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/vosk/script/raw2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/vosk_raw2text.py\" \"$@\"\n" }, { "path": "programs/asr/vosk/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\npython3 \"${base_dir}/bin/vosk_server.py\" --socketfile \"${socket_dir}/vosk.socket\" \"$@\"\n" }, { "path": "programs/asr/vosk/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/asr/vosk/script/wav2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/vosk_wav2text.py\" \"$@\"\n" }, { "path": "programs/asr/whisper/README.md", "content": "# Whisper\n\nSpeech to text service for Rhasspy based on [Whisper](https://github.com/openai/whisper).\n\nModels are downloaded automatically the first time they're used to the `config/data/asr/whisper` directory.\n\nAvailable models:\n\n* tiny.en\n* tiny\n* base.en\n* base\n* small.en\n* small\n* medium.en\n* medium\n* large-v1\n* large-v2\n* large\n\n## Installation\n\n1. Copy the contents of this directory to `config/programs/asr/whisper/`\n2. Run `script/setup`\n3. Test with `script/wav2text`\n * Example `script/wav2text 'tiny.en' /path/to/test.wav`\n" }, { "path": "programs/asr/whisper/bin/whisper_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport os\nimport socket\nimport threading\n\nimport numpy as np\nfrom whisper import Whisper, load_model, transcribe\n\n_LOGGER = logging.getLogger(\"whisper_server\")\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Name of Whisper model to use\")\n parser.add_argument(\n \"--language\",\n help=\"Whisper language\",\n )\n parser.add_argument(\"--device\", default=\"cpu\", choices=(\"cpu\", \"cuda\"))\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n _LOGGER.debug(\"Loading model: %s\", args.model)\n model = load_model(args.model, device=args.device)\n _LOGGER.info(\"Ready\")\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n\n # Start new thread for client\n threading.Thread(\n target=handle_client,\n args=(connection, model, args),\n daemon=True,\n ).start()\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\ndef handle_client(\n connection: socket.socket, model: Whisper, args: argparse.Namespace\n) -> None:\n try:\n is_first_audio = True\n\n with connection, connection.makefile(mode=\"rwb\") as conn_file:\n audio_bytes = bytes()\n while True:\n event_info = json.loads(conn_file.readline())\n event_type = event_info[\"type\"]\n\n if event_type == \"audio-chunk\":\n if is_first_audio:\n _LOGGER.debug(\"Receiving audio\")\n is_first_audio = False\n\n num_bytes = event_info[\"payload_length\"]\n audio_bytes += conn_file.read(num_bytes)\n elif event_type == \"audio-stop\":\n _LOGGER.debug(\"Audio stopped\")\n break\n\n audio_array = np.frombuffer(audio_bytes, dtype=np.int16)\n audio_array = audio_array.astype(np.float32) / 32768.0\n result = transcribe(model, audio_array, language=args.language)\n _LOGGER.debug(result)\n\n text = result[\"text\"]\n transcript_str = (\n json.dumps(\n {\"type\": \"transcript\", \"data\": {\"text\": text}}, ensure_ascii=False\n )\n + \"\\n\"\n )\n conn_file.write(transcript_str.encode())\n except Exception:\n _LOGGER.exception(\"Unexpected error in client thread\")\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/whisper/bin/whisper_wav2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\n\nfrom whisper import load_model, transcribe\n\n_LOGGER = logging.getLogger(\"whisper_wav2text\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Name of Whisper model to use\")\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path to WAV file(s) to transcribe\")\n parser.add_argument(\n \"--language\",\n help=\"Whisper language\",\n )\n parser.add_argument(\"--device\", default=\"cpu\", choices=(\"cpu\", \"cuda\"))\n #\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n _LOGGER.debug(\"Loading model: %s\", args.model)\n model = load_model(args.model, device=args.device)\n for wav_file in args.wav_file:\n _LOGGER.debug(\"Processing %s\", wav_file)\n result = transcribe(model, wav_file, language=args.language)\n _LOGGER.debug(result)\n\n text = result[\"text\"]\n print(text.strip())\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/whisper/requirements.txt", "content": "git+https://github.com/openai/whisper.git\n" }, { "path": "programs/asr/whisper/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\npython3 \"${base_dir}/bin/whisper_server.py\" --socketfile \"${socket_dir}/whisper.socket\" \"$@\"\n" }, { "path": "programs/asr/whisper/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/asr/whisper/script/wav2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/whisper_wav2text.py\" \"$@\"\n" }, { "path": "programs/asr/whisper-cpp/.gitignore", "content": "/build/\n" }, { "path": "programs/asr/whisper-cpp/Dockerfile.libwhisper", "content": "FROM debian:bullseye as build\nARG TARGETARCH\nARG TARGETVARIANT\n\nENV LANG C.UTF-8\nENV DEBIAN_FRONTEND=noninteractive\n\nRUN apt-get update && \\\n apt-get install --yes build-essential wget\n\nWORKDIR /build\n\nARG VERSION=1.1.0\nRUN wget \"https://github.com/ggerganov/whisper.cpp/archive/refs/tags/v${VERSION}.tar.gz\" && \\\n tar -xzf \"v${VERSION}.tar.gz\"\n\nRUN mv \"whisper.cpp-${VERSION}/\" 'whisper.cpp'\nCOPY lib/Makefile ./\nRUN cd \"whisper.cpp\" && make -j8\nRUN make\n\n# -----------------------------------------------------------------------------\n\nFROM scratch\n\nCOPY --from=build /build/libwhisper.so .\n" }, { "path": "programs/asr/whisper-cpp/Dockerfile.libwhisper.dockerignore", "content": "*\n!lib/Makefile\n" }, { "path": "programs/asr/whisper-cpp/README.md", "content": "# Whisper.cpp\n\nSpeech to text service for Rhasspy based on [whisper.cpp](https://github.com/ggerganov/whisper.cpp/).\n\nAdditional models can be downloaded here: https://huggingface.co/datasets/ggerganov/whisper.cpp\n\n## Installation\n\n1. Copy the contents of this directory to `config/programs/asr/whisper-cpp/`\n2. Run `script/setup.py`\n3. Download a model with `script/download.py`\n * Example: `script/download.py en_tiny`\n * Models are downloaded to `config/data/asr/whisper-cpp` directory\n4. Test with `script/wav2text`\n * Example `script/wav2text /path/to/ggml-tiny.en.bin /path/to/test.wav`\n" }, { "path": "programs/asr/whisper-cpp/bin/whisper_cpp_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport os\nimport socket\nimport threading\n\nimport numpy as np\nfrom whisper_cpp import Whisper\n\n_LOGGER = logging.getLogger(\"whisper_cpp_server\")\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to whisper.cpp model file\")\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n _LOGGER.debug(\"Loading model: %s\", args.model)\n with Whisper(args.model) as whisper:\n _LOGGER.info(\"Ready\")\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n\n # Start new thread for client\n threading.Thread(\n target=handle_client,\n args=(connection, whisper),\n daemon=True,\n ).start()\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\ndef handle_client(connection: socket.socket, whisper: Whisper) -> None:\n try:\n is_first_audio = True\n\n with connection, connection.makefile(mode=\"rwb\") as conn_file:\n audio_bytes = bytes()\n while True:\n event_info = json.loads(conn_file.readline())\n event_type = event_info[\"type\"]\n\n if event_type == \"audio-chunk\":\n if is_first_audio:\n _LOGGER.debug(\"Receiving audio\")\n is_first_audio = False\n\n num_bytes = event_info[\"payload_length\"]\n audio_bytes += conn_file.read(num_bytes)\n elif event_type == \"audio-stop\":\n _LOGGER.debug(\"Audio stopped\")\n break\n\n audio_array = np.frombuffer(audio_bytes, dtype=np.int16)\n audio_array = audio_array.astype(np.float32) / 32768.0\n text = \" \".join(whisper.transcribe(audio_array))\n _LOGGER.debug(text)\n\n transcript_str = (\n json.dumps(\n {\"type\": \"transcript\", \"data\": {\"text\": text}}, ensure_ascii=False\n )\n + \"\\n\"\n )\n conn_file.write(transcript_str.encode())\n except Exception:\n _LOGGER.exception(\"Unexpected error in client thread\")\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/whisper-cpp/bin/whisper_cpp_wav2text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport audioop\nimport logging\nimport wave\nfrom pathlib import Path\n\nimport numpy as np\nfrom whisper_cpp import Whisper\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to whisper.cpp model file\")\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path to WAV file(s) to transcribe\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n _LOGGER.debug(\"Loading model: %s\", args.model)\n\n with Whisper(args.model) as whisper:\n for wav_path in args.wav_file:\n wav_file: wave.Wave_read = wave.open(wav_path, \"rb\")\n with wav_file:\n rate = wav_file.getframerate()\n width = wav_file.getsampwidth()\n channels = wav_file.getnchannels()\n audio_bytes = wav_file.readframes(wav_file.getnframes())\n\n if width != 2:\n audio_bytes = audioop.lin2lin(audio_bytes, width, 2)\n\n if channels != 1:\n audio_bytes = audioop.tomono(audio_bytes, 2, 1.0, 1.0)\n\n if rate != 16000:\n audio_bytes, _state = audioop.ratecv(\n audio_bytes, 2, 1, rate, 16000, None\n )\n\n audio_array = np.frombuffer(audio_bytes, dtype=np.int16)\n audio_array = audio_array.astype(np.float32) / 32768.0\n\n text = \" \".join(whisper.transcribe(audio_array))\n print(text)\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/whisper-cpp/lib/Makefile", "content": "UNAME_M := $(shell uname -m)\n\nCFLAGS = -Iwhisper.cpp -O3 -std=c11 -fPIC\nCXXFLAGS = -Iwhisper.cpp -O3 -std=c++11 -fPIC --shared -static-libstdc++\nLDFLAGS =\n\n# Linux\nCFLAGS += -pthread\nCXXFLAGS += -pthread\n\nAVX1_M := $(shell grep \"avx \" /proc/cpuinfo)\nifneq (,$(findstring avx,$(AVX1_M)))\n\tCFLAGS += -mavx\nendif\nAVX2_M := $(shell grep \"avx2 \" /proc/cpuinfo)\nifneq (,$(findstring avx2,$(AVX2_M)))\n\tCFLAGS += -mavx2\nendif\nFMA_M := $(shell grep \"fma \" /proc/cpuinfo)\nifneq (,$(findstring fma,$(FMA_M)))\n\tCFLAGS += -mfma\nendif\nF16C_M := $(shell grep \"f16c \" /proc/cpuinfo)\nifneq (,$(findstring f16c,$(F16C_M)))\n\tCFLAGS += -mf16c\nendif\nSSE3_M := $(shell grep \"sse3 \" /proc/cpuinfo)\nifneq (,$(findstring sse3,$(SSE3_M)))\n\tCFLAGS += -msse3\nendif\n\n# amd64 and arm64 only\nifeq ($(UNAME_M),amd64)\n\tCFLAGS += -mavx -mavx2 -mfma -mf16c\nendif\n\nifneq ($(filter armv8%,$(UNAME_M)),)\n\t# Raspberry Pi 4\n\tCFLAGS += -mfp16-format=ieee -mno-unaligned-access\nendif\n\ndefault: libwhisper.so\n\nlibwhisper.so: whisper.cpp/ggml.o whisper.cpp/whisper.cpp\n\t$(CXX) $(CXXFLAGS) $^ -o $@ $(LDFLAGS)\n" }, { "path": "programs/asr/whisper-cpp/lib/whisper_cpp.py", "content": "import ctypes\nfrom pathlib import Path\nfrom typing import Iterable, Union\n\nimport numpy as np\n\n\n# Must match struct in whisper.h\nclass WhisperFullParams(ctypes.Structure):\n _fields_ = [\n (\"strategy\", ctypes.c_int),\n #\n (\"n_max_text_ctx\", ctypes.c_int),\n (\"n_threads\", ctypes.c_int),\n (\"offset_ms\", ctypes.c_int),\n (\"duration_ms\", ctypes.c_int),\n #\n (\"translate\", ctypes.c_bool),\n (\"no_context\", ctypes.c_bool),\n (\"single_segment\", ctypes.c_bool),\n (\"print_special\", ctypes.c_bool),\n (\"print_progress\", ctypes.c_bool),\n (\"print_realtime\", ctypes.c_bool),\n (\"print_timestamps\", ctypes.c_bool),\n #\n (\"token_timestamps\", ctypes.c_bool),\n (\"thold_pt\", ctypes.c_float),\n (\"thold_ptsum\", ctypes.c_float),\n (\"max_len\", ctypes.c_int),\n (\"max_tokens\", ctypes.c_int),\n #\n (\"speed_up\", ctypes.c_bool),\n (\"audio_ctx\", ctypes.c_int),\n #\n (\"prompt_tokens\", ctypes.c_void_p),\n (\"prompt_n_tokens\", ctypes.c_int),\n #\n (\"language\", ctypes.c_char_p),\n #\n (\"suppress_blank\", ctypes.c_bool),\n #\n (\"temperature_inc\", ctypes.c_float),\n (\"entropy_thold\", ctypes.c_float),\n (\"logprob_thold\", ctypes.c_float),\n (\"no_speech_thold\", ctypes.c_float),\n #\n (\"greedy\", ctypes.c_int * 1),\n (\"beam_search\", ctypes.c_int * 3),\n #\n (\"new_segment_callback\", ctypes.c_void_p),\n (\"new_segment_callback_user_data\", ctypes.c_void_p),\n #\n (\"encoder_begin_callback\", ctypes.c_void_p),\n (\"encoder_begin_callback_user_data\", ctypes.c_void_p),\n ]\n\n\nclass WhisperError(Exception):\n pass\n\n\nclass Whisper:\n def __init__(\n self,\n model_path: Union[str, Path],\n libwhisper_path: Union[str, Path] = \"libwhisper.so\",\n language: str = \"en\",\n ):\n self.model_path = Path(model_path)\n self.whisper = ctypes.CDLL(str(libwhisper_path))\n\n # Set return types\n self.whisper.whisper_init_from_file.restype = ctypes.c_void_p\n self.whisper.whisper_full_default_params.restype = WhisperFullParams\n self.whisper.whisper_full_get_segment_text.restype = ctypes.c_char_p\n\n # initialize whisper.cpp context\n filename_bytes = str(self.model_path.absolute()).encode(\"utf-8\")\n self.ctx = self.whisper.whisper_init_from_file(filename_bytes)\n\n # get default whisper parameters and adjust as needed\n self.params = self.whisper.whisper_full_default_params()\n self.params.print_realtime = False\n self.params.print_special = False\n self.params.print_progress = False\n self.params.print_timestamps = False\n self.params.language = language.encode(\"utf-8\")\n\n def transcribe(self, audio_array: np.ndarray) -> Iterable[str]:\n \"\"\"Transcribe float32 audio in [0, 1] to text.\"\"\"\n result = self.whisper.whisper_full(\n ctypes.c_void_p(self.ctx),\n self.params,\n audio_array.ctypes.data_as(ctypes.POINTER(ctypes.c_float)),\n len(audio_array),\n )\n if result != 0:\n raise WhisperError(str(result))\n\n num_segments = self.whisper.whisper_full_n_segments(ctypes.c_void_p(self.ctx))\n for i in range(num_segments):\n text_bytes = self.whisper.whisper_full_get_segment_text(\n ctypes.c_void_p(self.ctx), i\n )\n yield text_bytes.decode(\"utf-8\")\n\n def __enter__(self):\n return self\n\n def __exit__(self, exc_type, exc_value, traceback):\n self.whisper.whisper_free(ctypes.c_void_p(self.ctx))\n self.whisper = None\n self.ctx = None\n" }, { "path": "programs/asr/whisper-cpp/requirements.txt", "content": "numpy\n" }, { "path": "programs/asr/whisper-cpp/script/build_libwhisper", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\ndocker buildx build \"${base_dir}\" \\\n -f \"${base_dir}/Dockerfile.libwhisper\" \\\n --platform 'linux/amd64,linux/arm64' \\\n --output \"type=local,dest=${base_dir}/build/\"\n\n# -----------------------------------------------------------------------------\n\necho \"Copy the appropriate libwhisper.so from build/ to lib/\"\n" }, { "path": "programs/asr/whisper-cpp/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\nMODELS = [\"tiny.en\", \"base.en\"]\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"model\",\n nargs=\"+\",\n choices=MODELS,\n help=\"Pocketsphinx model(s) to download\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: share)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/asr_whisper-cpp-ggml-{model}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if args.destination:\n args.destination = Path(args.destination)\n else:\n # Assume we're in programs/asr/whisper-cpp/script\n data_dir = _DIR.parent.parent.parent.parent / \"data\"\n args.destination = data_dir / \"asr\" / \"whisper-cpp\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n for model in args.model:\n url = args.link_format.format(model=model)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/whisper-cpp/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\nlib_dir=\"${base_dir}/lib\"\nexport LD_LIBRARY_PATH=\"${lib_dir}:${LD_LIBRARY_PATH}\"\nexport PYTHONPATH=\"${lib_dir}:${PYTHONPATH}\"\n\npython3 \"${base_dir}/bin/whisper_cpp_server.py\" --socketfile \"${socket_dir}/whisper-cpp.socket\" \"$@\"\n" }, { "path": "programs/asr/whisper-cpp/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\npython3 \"${this_dir}/setup.py\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/asr/whisper-cpp/script/setup.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport platform\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\nPLATFORMS = {\"x86_64\": \"amd64\", \"aarch64\": \"arm64\"}\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--platform\",\n help=\"CPU architecture to download (amd64, arm64)\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: lib)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/libwhisper_{platform}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if not args.platform:\n args.platform = platform.machine()\n\n args.platform = PLATFORMS.get(args.platform, args.platform)\n\n if not args.destination:\n args.destination = _DIR.parent / \"lib\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n url = args.link_format.format(platform=args.platform)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/asr/whisper-cpp/script/wav2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nlib_dir=\"${base_dir}/lib\"\nexport LD_LIBRARY_PATH=\"${lib_dir}:${LD_LIBRARY_PATH}\"\nexport PYTHONPATH=\"${lib_dir}:${PYTHONPATH}\"\n\npython3 \"${base_dir}/bin/whisper_cpp_wav2text.py\" \"$@\"\n" }, { "path": "programs/handle/date_time/bin/date_time.py", "content": "#!/usr/bin/env python3\nimport re\nimport sys\nfrom datetime import datetime\n\n\ndef main() -> None:\n text = sys.stdin.read().strip().lower()\n words = [re.sub(r\"\\W\", \"\", word) for word in text.split()]\n\n now = datetime.now()\n if \"time\" in words:\n print(now.strftime(\"%I:%M %p\"))\n elif \"date\" in words:\n print(now.strftime(\"%A, %B %d, %Y\"))\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/handle/home_assistant/bin/converse.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport sys\nfrom pathlib import Path\nfrom urllib.request import Request, urlopen\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"url\",\n help=\"URL of API endpoint\",\n )\n parser.add_argument(\"token_file\", help=\"Path to file with authorization token\")\n parser.add_argument(\"--language\", help=\"Language code to use\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n token = Path(args.token_file).read_text(encoding=\"utf-8\").strip()\n headers = {\"Authorization\": f\"Bearer {token}\"}\n\n data_dict = {\"text\": sys.stdin.read()}\n if args.language:\n data_dict[\"language\"] = args.language\n\n data = json.dumps(data_dict, ensure_ascii=False).encode(\"utf-8\")\n request = Request(args.url, data=data, headers=headers)\n\n with urlopen(request) as response:\n response = json.loads(response.read())\n response_text = (\n response.get(\"response\", {})\n .get(\"speech\", {})\n .get(\"plain\", {})\n .get(\"speech\", \"\")\n )\n print(response_text)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/intent/regex/bin/regex.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport re\nfrom collections import defaultdict\nfrom typing import Dict, List, Optional\n\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.intent import Entity, Intent, NotRecognized, Recognize\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-i\",\n \"--intent\",\n required=True,\n nargs=2,\n metavar=(\"name\", \"regex\"),\n action=\"append\",\n default=[],\n help=\"Intent name and regex\",\n )\n args = parser.parse_args()\n\n # intent name -> [pattern]\n patterns: Dict[str, List[re.Pattern]] = defaultdict(list)\n\n for intent_name, pattern_str in args.intent:\n patterns[intent_name].append(re.compile(pattern_str, re.IGNORECASE))\n\n try:\n while True:\n event = read_event()\n if event is None:\n break\n\n if Recognize.is_type(event.type):\n recognize = Recognize.from_event(event)\n text = _clean(recognize.text)\n intent = _recognize(text, patterns)\n if intent is None:\n write_event(NotRecognized().event())\n else:\n write_event(intent.event())\n except KeyboardInterrupt:\n pass\n\n\ndef _clean(text: str) -> str:\n text = \" \".join(text.split())\n return text\n\n\ndef _recognize(text: str, patterns: Dict[str, List[re.Pattern]]) -> Optional[Intent]:\n for intent_name, intent_patterns in patterns.items():\n for intent_pattern in intent_patterns:\n match = intent_pattern.match(text)\n if match is None:\n continue\n\n return Intent(\n name=intent_name,\n entities=[\n Entity(name=name, value=value)\n for name, value in match.groupdict().items()\n ],\n )\n\n return None\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/mic/pyaudio/README.md", "content": "# PyAudio\n\nAudio input service for Rhasspy based on [PyAudio](https://people.csail.mit.edu/hubert/pyaudio/docs/).\n" }, { "path": "programs/mic/pyaudio/bin/pyaudio_events.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nfrom pathlib import Path\n\nfrom pyaudio_shared import iter_chunks\n\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n DEFAULT_SAMPLES_PER_CHUNK,\n AudioChunk,\n)\nfrom rhasspy3.event import write_event\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--rate\", type=int, default=DEFAULT_IN_RATE, help=\"Sample rate (hertz)\"\n )\n parser.add_argument(\n \"--width\", type=int, default=DEFAULT_IN_WIDTH, help=\"Sample width (bytes)\"\n )\n parser.add_argument(\n \"--channels\", type=int, default=DEFAULT_IN_CHANNELS, help=\"Sample channel count\"\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=DEFAULT_SAMPLES_PER_CHUNK,\n help=\"Number of samples to process at a time\",\n )\n parser.add_argument(\"--device\", help=\"Name or index of device to use\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n for chunk in iter_chunks(\n args.device, args.rate, args.width, args.channels, args.samples_per_chunk\n ):\n write_event(AudioChunk(args.rate, args.width, args.channels, chunk).event())\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/mic/pyaudio/bin/pyaudio_list_mics.py", "content": "#!/usr/bin/env python3\n\nimport pyaudio\n\n\ndef main() -> None:\n audio_system = pyaudio.PyAudio()\n for i in range(audio_system.get_device_count()):\n print(audio_system.get_device_info_by_index(i))\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/mic/pyaudio/bin/pyaudio_raw.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport sys\nfrom pathlib import Path\n\nfrom pyaudio_shared import iter_chunks\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--rate\", type=int, required=True, help=\"Sample rate (hertz)\")\n parser.add_argument(\"--width\", type=int, required=True, help=\"Sample width (bytes)\")\n parser.add_argument(\n \"--channels\", type=int, required=True, help=\"Sample channel count\"\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n required=True,\n help=\"Number of samples to process at a time\",\n )\n parser.add_argument(\"--device\", help=\"Name or index of device to use\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n for chunk in iter_chunks(\n args.device, args.rate, args.width, args.channels, args.samples_per_chunk\n ):\n sys.stdout.buffer.write(chunk)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/mic/pyaudio/bin/pyaudio_shared.py", "content": "import logging\nfrom pathlib import Path\nfrom typing import Iterable, Optional, Union\n\nimport pyaudio\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef iter_chunks(\n device: Optional[Union[int, str]],\n rate: int,\n width: int,\n channels: int,\n samples_per_chunk: int,\n) -> Iterable[bytes]:\n \"\"\"Open input stream and yield audio chunks.\"\"\"\n audio_system = pyaudio.PyAudio()\n try:\n if isinstance(device, str):\n try:\n device = int(device)\n except ValueError:\n for i in range(audio_system.get_device_count()):\n info = audio_system.get_device_info_by_index(i)\n if device == info[\"name\"]:\n device = i\n break\n\n assert device is not None, f\"No device named: {device}\"\n\n _LOGGER.debug(\"Device: %s\", device)\n stream = audio_system.open(\n input_device_index=device,\n format=audio_system.get_format_from_width(width),\n channels=channels,\n rate=rate,\n input=True,\n frames_per_buffer=samples_per_chunk,\n )\n\n chunk = stream.read(samples_per_chunk)\n while chunk:\n yield chunk\n chunk = stream.read(samples_per_chunk)\n except KeyboardInterrupt:\n pass\n finally:\n audio_system.terminate()\n" }, { "path": "programs/mic/pyaudio/requirements.txt", "content": "pyaudio\n" }, { "path": "programs/mic/pyaudio/script/events", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/pyaudio_events.py\" \"$@\"\n" }, { "path": "programs/mic/pyaudio/script/list_mics", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/pyaudio_list_mics.py\" \"$@\"\n" }, { "path": "programs/mic/pyaudio/script/raw", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/pyaudio_raw.py\" \"$@\"\n" }, { "path": "programs/mic/pyaudio/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# Install rhasspy3\nrhasspy3_dir=\"${base_dir}/../../../..\"\npip3 install -e \"${rhasspy3_dir}\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/mic/sounddevice/README.md", "content": "# sounddevice\n\nAudio input service for Rhasspy based on [sounddevice](https://python-sounddevice.readthedocs.io).\n" }, { "path": "programs/mic/sounddevice/bin/sounddevice_events.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nfrom pathlib import Path\n\nfrom sounddevice_shared import iter_chunks\n\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n DEFAULT_SAMPLES_PER_CHUNK,\n AudioChunk,\n)\nfrom rhasspy3.event import write_event\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--rate\", type=int, default=DEFAULT_IN_RATE, help=\"Sample rate (hertz)\"\n )\n parser.add_argument(\n \"--width\", type=int, default=DEFAULT_IN_WIDTH, help=\"Sample width (bytes)\"\n )\n parser.add_argument(\n \"--channels\", type=int, default=DEFAULT_IN_CHANNELS, help=\"Sample channel count\"\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n default=DEFAULT_SAMPLES_PER_CHUNK,\n help=\"Number of samples to process at a time\",\n )\n parser.add_argument(\"--device\", help=\"Name or index of device to use\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n for chunk in iter_chunks(\n args.device, args.rate, args.width, args.channels, args.samples_per_chunk\n ):\n write_event(AudioChunk(args.rate, args.width, args.channels, chunk).event())\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/mic/sounddevice/bin/sounddevice_list_mics.py", "content": "#!/usr/bin/env python3\n\nimport sounddevice\n\n\ndef main() -> None:\n for info in sounddevice.query_devices():\n print(info)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/mic/sounddevice/bin/sounddevice_raw.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport sys\nfrom pathlib import Path\n\nfrom sounddevice_shared import iter_chunks\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--rate\", type=int, required=True, help=\"Sample rate (hertz)\")\n parser.add_argument(\"--width\", type=int, required=True, help=\"Sample width (bytes)\")\n parser.add_argument(\n \"--channels\", type=int, required=True, help=\"Sample channel count\"\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n type=int,\n required=True,\n help=\"Number of samples to process at a time\",\n )\n parser.add_argument(\"--device\", help=\"Name or index of device to use\")\n #\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n for chunk in iter_chunks(\n args.device, args.rate, args.width, args.channels, args.samples_per_chunk\n ):\n sys.stdout.buffer.write(chunk)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/mic/sounddevice/bin/sounddevice_shared.py", "content": "import logging\nfrom pathlib import Path\nfrom typing import Iterable, Optional, Union\n\nimport sounddevice\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef iter_chunks(\n device: Optional[Union[int, str]],\n rate: int,\n width: int,\n channels: int,\n samples_per_chunk: int,\n) -> Iterable[bytes]:\n \"\"\"Open input stream and yield audio chunks.\"\"\"\n try:\n if isinstance(device, str):\n try:\n device = int(device)\n except ValueError:\n for i, info in enumerate(sounddevice.query_devices()):\n if device == info[\"name\"]:\n device = i\n break\n\n assert device is not None, f\"No device named: {device}\"\n\n _LOGGER.debug(\"Device: %s\", device)\n\n with sounddevice.RawInputStream(\n samplerate=rate,\n blocksize=samples_per_chunk,\n device=device,\n channels=channels,\n dtype=\"int16\",\n ) as stream:\n chunk, _overflowed = stream.read(samples_per_chunk)\n while chunk:\n yield chunk\n chunk, _overflowed = stream.read(samples_per_chunk)\n except KeyboardInterrupt:\n pass\n" }, { "path": "programs/mic/sounddevice/requirements.txt", "content": "sounddevice\n" }, { "path": "programs/mic/sounddevice/script/events", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/sounddevice_events.py\" \"$@\"\n" }, { "path": "programs/mic/sounddevice/script/list_mics", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 -m sounddevice\n" }, { "path": "programs/mic/sounddevice/script/raw", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/sounddevice_raw.py\" \"$@\"\n" }, { "path": "programs/mic/sounddevice/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# Install rhasspy3\nrhasspy3_dir=\"${base_dir}/../../../..\"\npip3 install -e \"${rhasspy3_dir}\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/mic/udp_raw/bin/udp_raw.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport socketserver\nfrom functools import partial\n\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n AudioChunk,\n)\nfrom rhasspy3.event import write_event\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--port\", type=int, required=True)\n parser.add_argument(\"--host\", default=\"0.0.0.0\")\n #\n parser.add_argument(\n \"--rate\", type=int, default=DEFAULT_IN_RATE, help=\"Sample rate (hertz)\"\n )\n parser.add_argument(\n \"--width\", type=int, default=DEFAULT_IN_WIDTH, help=\"Sample width (bytes)\"\n )\n parser.add_argument(\n \"--channels\",\n type=int,\n default=DEFAULT_IN_CHANNELS,\n help=\"Sample channel count\",\n )\n args = parser.parse_args()\n\n with socketserver.UDPServer(\n (args.host, args.port),\n partial(MicUDPHandler, args.rate, args.width, args.channels),\n ) as server:\n server.serve_forever()\n\n\nclass MicUDPHandler(socketserver.BaseRequestHandler):\n def __init__(self, rate: int, width: int, channels: int, *args, **kwargs):\n self.rate = rate\n self.width = width\n self.channels = channels\n self.state = None\n super().__init__(*args, **kwargs)\n\n def handle(self):\n audio_bytes = self.request[0]\n write_event(\n AudioChunk(\n rate=self.rate,\n width=self.width,\n channels=self.channels,\n audio=audio_bytes,\n ).event()\n )\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/remote/websocket/bin/stream2stream.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport asyncio\nimport json\nimport logging\nimport sys\nfrom pathlib import Path\nfrom typing import Optional\n\nfrom websockets import connect\nfrom websockets.exceptions import ConnectionClosedOK\n\nfrom rhasspy3.audio import AudioChunk, AudioStart, AudioStop\nfrom rhasspy3.event import Event, read_event, write_event\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"uri\")\n args = parser.parse_args()\n\n async with connect(args.uri) as websocket:\n recv_task = asyncio.create_task(websocket.recv())\n pending = {recv_task}\n\n while True:\n mic_event = read_event()\n if mic_event is None:\n break\n\n if not AudioChunk.is_type(mic_event.type):\n continue\n\n mic_chunk = AudioChunk.from_event(mic_event)\n send_task = asyncio.create_task(websocket.send(mic_chunk.audio))\n pending.add(send_task)\n\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n if recv_task in done:\n for task in pending:\n task.cancel()\n break\n\n if send_task not in done:\n await send_task\n\n try:\n while True:\n start: Optional[AudioStart] = None\n data = await websocket.recv()\n try:\n event = Event.from_dict(json.loads(data))\n if AudioStart.is_type(event.type):\n start = AudioStart.from_event(event)\n break\n except Exception:\n continue\n\n assert start is not None\n\n while True:\n data = await websocket.recv()\n if isinstance(data, bytes):\n write_event(\n AudioChunk(\n start.rate,\n start.width,\n start.channels,\n data,\n ).event()\n )\n else:\n try:\n event = Event.from_dict(json.loads(data))\n if AudioStop.is_type(event.type):\n break\n except Exception:\n pass\n except ConnectionClosedOK:\n pass\n finally:\n write_event(AudioStop().event())\n\n\nasync def play(websocket, done_event: asyncio.Event):\n try:\n while True:\n audio_bytes = await websocket.recv()\n if isinstance(audio_bytes, bytes):\n done_event.set()\n sys.stdout.buffer.write(audio_bytes)\n sys.stdout.buffer.flush()\n except ConnectionClosedOK:\n pass\n\n\nif __name__ == \"__main__\":\n try:\n asyncio.run(main())\n except KeyboardInterrupt:\n pass\n" }, { "path": "programs/remote/websocket/requirements.txt", "content": "websockets\n" }, { "path": "programs/remote/websocket/script/run", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/stream2stream.py\" \"$@\"\n" }, { "path": "programs/remote/websocket/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# Install rhasspy3\nrhasspy3_dir=\"${base_dir}/../../../..\"\npip3 install -e \"${rhasspy3_dir}\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/snd/udp_raw/bin/udp_raw.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport socket\n\nfrom rhasspy3.audio import (\n DEFAULT_OUT_CHANNELS,\n DEFAULT_OUT_RATE,\n DEFAULT_OUT_WIDTH,\n AudioChunk,\n AudioChunkConverter,\n AudioStop,\n)\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.snd import Played\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--port\", type=int, required=True)\n parser.add_argument(\"--host\", required=True)\n #\n parser.add_argument(\n \"--rate\", type=int, default=DEFAULT_OUT_RATE, help=\"Sample rate (hertz)\"\n )\n parser.add_argument(\n \"--width\", type=int, default=DEFAULT_OUT_WIDTH, help=\"Sample width (bytes)\"\n )\n parser.add_argument(\n \"--channels\",\n type=int,\n default=DEFAULT_OUT_CHANNELS,\n help=\"Sample channel count\",\n )\n #\n args = parser.parse_args()\n\n converter = AudioChunkConverter(args.rate, args.width, args.channels)\n sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)\n while True:\n event = read_event()\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n chunk = converter.convert(chunk)\n sock.sendto(chunk.audio, (args.host, args.port))\n elif AudioStop.is_type(event.type):\n break\n\n write_event(Played().event())\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/tts/coqui-tts/README.md", "content": "# Coqui-TTS\n\nText to speech service for Rhasspy based on [Coqui-TTS](https://tts.readthedocs.io).\n" }, { "path": "programs/tts/coqui-tts/requirements.txt", "content": "tts\n" }, { "path": "programs/tts/coqui-tts/script/list_models", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\ntts --list_models \"$@\"\n" }, { "path": "programs/tts/coqui-tts/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\ntts-server --model_name 'tts_models/en/ljspeech/vits' \"$@\"\n" }, { "path": "programs/tts/coqui-tts/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/tts/flite/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"download\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"language\",\n nargs=\"+\",\n choices=(\"en\", \"indic\"),\n help=\"Voice language(s) to download\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: share)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/tts_flite-{language}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if not args.destination:\n args.destination = _DIR.parent / \"share\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n for language in args.language:\n url = args.link_format.format(language=language)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/tts/flite/script/setup", "content": "#!/usr/bin/env bash\nsudo apt-get update\nsudo apt-get install flite\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/tts/larynx/README.md", "content": "# Larynx\n\nText to speech service for Rhasspy based on [Larynx](https://github.com/rhasspy/larynx/).\n" }, { "path": "programs/tts/larynx/bin/larynx_client.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport shutil\nimport sys\nfrom pathlib import Path\nfrom urllib.parse import urlencode\nfrom urllib.request import urlopen\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"url\",\n help=\"URL of API endpoint\",\n )\n parser.add_argument(\"voice\", help=\"VOICE parameter\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n params = {\"INPUT_TEXT\": sys.stdin.read(), \"VOICE\": args.voice}\n url = args.url + \"?\" + urlencode(params)\n\n _LOGGER.debug(url)\n\n with urlopen(url) as response:\n shutil.copyfileobj(response, sys.stdout.buffer)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/tts/larynx/requirements.txt", "content": "larynx\n" }, { "path": "programs/tts/larynx/script/list_models", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nlarynx --list \"$@\"\n" }, { "path": "programs/tts/larynx/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nlarynx-server \"$@\"\n" }, { "path": "programs/tts/larynx/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\" \\\n 'jinja2<3.1.0' \\\n -f 'https://synesthesiam.github.io/prebuilt-apps/' \\\n -f 'https://download.pytorch.org/whl/cpu/torch_stable.html'\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/tts/marytts/bin/marytts.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport shutil\nimport sys\nfrom pathlib import Path\nfrom urllib.parse import urlencode\nfrom urllib.request import urlopen\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"url\",\n help=\"URL of API endpoint\",\n )\n parser.add_argument(\"voice\", help=\"VOICE parameter\")\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n params = {\"INPUT_TEXT\": sys.stdin.read(), \"VOICE\": args.voice}\n url = args.url + \"?\" + urlencode(params)\n\n _LOGGER.debug(url)\n\n with urlopen(url) as response:\n shutil.copyfileobj(response, sys.stdout.buffer)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/tts/mimic3/README.md", "content": "# Mimic 3\n\nText to speech service for Rhasspy based on [Mimic 3](https://github.com/mycroftAI/mimic3).\n" }, { "path": "programs/tts/mimic3/bin/mimic3_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport os\nimport socket\nimport threading\nfrom pathlib import Path\n\nfrom mimic3_tts import (\n DEFAULT_VOICE,\n AudioResult,\n Mimic3Settings,\n Mimic3TextToSpeechSystem,\n)\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"voices_dir\", help=\"Path to directory with /\")\n parser.add_argument(\"--voice\", default=DEFAULT_VOICE, help=\"Name of voice to use\")\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n mimic3 = Mimic3TextToSpeechSystem(\n Mimic3Settings(\n voices_directories=[args.voices_dir],\n voices_download_dir=args.voices_dir,\n )\n )\n\n if \"#\" in args.voice:\n # Case to handle a multi-speaker voice definition\n voice, _speaker = args.voice.split(\"#\", maxsplit=1)\n _LOGGER.debug(\"Preloading voice: %s\", voice)\n mimic3.preload_voice(voice)\n else:\n _LOGGER.debug(\"Preloading voice: %s\", args.voice)\n mimic3.preload_voice(args.voice)\n\n _LOGGER.info(\"Ready\")\n\n mimic3.voice = args.voice\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n\n # Start new thread for client\n threading.Thread(\n target=handle_client,\n args=(connection, mimic3),\n daemon=True,\n ).start()\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\ndef handle_client(connection: socket.socket, mimic3: Mimic3TextToSpeechSystem) -> None:\n try:\n with connection, connection.makefile(mode=\"rwb\") as conn_file:\n while True:\n event_info = json.loads(conn_file.readline())\n event_type = event_info[\"type\"]\n\n if event_type == \"synthesize\":\n text = event_info[\"data\"][\"text\"]\n _LOGGER.debug(\"synthesize: text='%s'\", text)\n\n mimic3.begin_utterance()\n mimic3.speak_text(text)\n results = mimic3.end_utterance()\n\n is_first_audio = True\n for result in results:\n if not isinstance(result, AudioResult):\n continue\n\n data = {\n \"rate\": result.sample_rate_hz,\n \"width\": result.sample_width_bytes,\n \"channels\": result.num_channels,\n }\n\n if is_first_audio:\n is_first_audio = False\n conn_file.write(\n (\n json.dumps({\"type\": \"audio-start\", \"data\": data})\n + \"\\n\"\n ).encode()\n )\n\n conn_file.write(\n (\n json.dumps(\n {\n \"type\": \"audio-chunk\",\n \"data\": data,\n \"payload_length\": len(result.audio_bytes),\n }\n )\n + \"\\n\"\n ).encode()\n )\n conn_file.write(result.audio_bytes)\n\n conn_file.write(\n (\n json.dumps({\"type\": \"audio-stop\"}, ensure_ascii=False)\n + \"\\n\"\n ).encode()\n )\n break\n except Exception:\n _LOGGER.exception(\"Unexpected error in client thread\")\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/tts/mimic3/requirements.txt", "content": "mycroft-mimic3-tts[all]\n" }, { "path": "programs/tts/mimic3/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\npython3 \"${base_dir}/bin/mimic3_server.py\" --socketfile \"${socket_dir}/mimic3.socket\" \"$@\"\n" }, { "path": "programs/tts/mimic3/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\nmimic3-download --output-dir \"${base_dir}/share\" 'apope'\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/tts/piper/README.md", "content": "# Piper\n\nText to speech service for Rhasspy based on [Piper](https://github.com/rhasspy/piper).\n\n\n## Installation\n\n1. Copy the contents of this directory to `config/programs/tts/piper/`\n2. Run `script/setup`\n3. Download a model with `script/download.py`\n * Example: `script/download.py english`\n * Models are downloaded to `config/data/tts/piper` directory\n4. Test with `bin/piper`\n * Example `echo 'Welcome to the world of speech synthesis.' | bin/piper --model /path/to/en-us-blizzard_lessac-medium.onnx --output_file welcome.wav`\n" }, { "path": "programs/tts/piper/bin/piper_server.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport json\nimport logging\nimport os\nimport socket\nimport subprocess\nimport tempfile\nimport wave\nfrom pathlib import Path\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to model file (.onnx)\")\n parser.add_argument(\n \"--socketfile\", required=True, help=\"Path to Unix domain socket file\"\n )\n parser.add_argument(\n \"--auto-punctuation\", default=\".?!\", help=\"Automatically add punctuation\"\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n # Need to unlink socket if it exists\n try:\n os.unlink(args.socketfile)\n except OSError:\n pass\n\n try:\n # Create socket server\n sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)\n sock.bind(args.socketfile)\n sock.listen()\n\n with tempfile.TemporaryDirectory() as temp_dir:\n command = [\n str(_DIR / \"piper\"),\n \"--model\",\n str(args.model),\n \"--output_dir\",\n temp_dir,\n ]\n with subprocess.Popen(\n command,\n stdin=subprocess.PIPE,\n stdout=subprocess.PIPE,\n universal_newlines=True,\n ) as proc:\n _LOGGER.info(\"Ready\")\n\n # Listen for connections\n while True:\n try:\n connection, client_address = sock.accept()\n _LOGGER.debug(\"Connection from %s\", client_address)\n handle_connection(connection, proc, args)\n except KeyboardInterrupt:\n break\n except Exception:\n _LOGGER.exception(\"Error communicating with socket client\")\n finally:\n os.unlink(args.socketfile)\n\n\ndef handle_connection(\n connection: socket.socket, proc: subprocess.Popen, args: argparse.Namespace\n) -> None:\n assert proc.stdin is not None\n assert proc.stdout is not None\n\n with connection, connection.makefile(mode=\"rwb\") as conn_file:\n while True:\n event_info = json.loads(conn_file.readline())\n event_type = event_info[\"type\"]\n\n if event_type != \"synthesize\":\n continue\n\n raw_text = event_info[\"data\"][\"text\"]\n text = raw_text.strip()\n if args.auto_punctuation and text:\n has_punctuation = False\n for punc_char in args.auto_punctuation:\n if text[-1] == punc_char:\n has_punctuation = True\n break\n\n if not has_punctuation:\n text = text + args.auto_punctuation[0]\n\n _LOGGER.debug(\"synthesize: raw_text=%s, text='%s'\", raw_text, text)\n\n # Text in, file path out\n print(text.strip(), file=proc.stdin, flush=True)\n output_path = proc.stdout.readline().strip()\n _LOGGER.debug(output_path)\n\n wav_file: wave.Wave_read = wave.open(output_path, \"rb\")\n with wav_file:\n data = {\n \"rate\": wav_file.getframerate(),\n \"width\": wav_file.getsampwidth(),\n \"channels\": wav_file.getnchannels(),\n }\n\n conn_file.write(\n (\n json.dumps(\n {\"type\": \"audio-start\", \"data\": data}, ensure_ascii=False\n )\n + \"\\n\"\n ).encode()\n )\n\n # Audio\n audio_bytes = wav_file.readframes(wav_file.getnframes())\n conn_file.write(\n (\n json.dumps(\n {\n \"type\": \"audio-chunk\",\n \"data\": data,\n \"payload_length\": len(audio_bytes),\n },\n ensure_ascii=False,\n )\n + \"\\n\"\n ).encode()\n )\n conn_file.write(audio_bytes)\n\n conn_file.write(\n (json.dumps({\"type\": \"audio-stop\"}, ensure_ascii=False) + \"\\n\").encode()\n )\n os.unlink(output_path)\n break\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/tts/piper/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\n_VOICE_NAMES = [\n \"ca-upc_ona-x-low\",\n \"ca-upc_pau-x-low\",\n \"da-nst_talesyntese-medium\",\n \"de-eva_k-x-low\",\n \"de-thorsten-low\",\n \"en-gb-alan-low\",\n \"en-gb-southern_english_female-low\",\n \"en-us-amy-low\",\n \"en-us-kathleen-low\",\n \"en-us-lessac-low\",\n \"en-us-lessac-medium\",\n \"en-us-libritts-high\",\n \"en-us-ryan-high\",\n \"en-us-ryan-low\",\n \"en-us-ryan-medium\",\n \"es-carlfm-x-low\",\n \"fi-harri-low\",\n \"fr-siwis-low\",\n \"fr-siwis-medium\",\n \"it-riccardo_fasol-x-low\",\n \"kk-iseke-x-low\",\n \"kk-issai-high\",\n \"kk-raya-x-low\",\n \"ne-google-medium\",\n \"ne-google-x-low\",\n \"nl-mls_7432-low\",\n \"nl-nathalie-x-low\",\n \"nl-rdh-medium\",\n \"nl-rdh-x-low\",\n \"no-talesyntese-medium\",\n \"pl-mls_6892-low\",\n \"pt-br-edresson-low\",\n \"uk-lada-x-low\",\n \"vi-25hours-single-low\",\n \"vi-vivos-x-low\",\n \"zh-cn-huayan-x-low\",\n]\n\n_VOICES = {\n \"catalan\": \"ca\",\n \"ca\": \"ca-upc_ona-x-low\",\n #\n \"danish\": \"da\",\n \"da\": \"da-nst_talesyntese-medium\",\n #\n \"german\": \"de\",\n \"de\": \"de-thorsten-low\",\n #\n \"english\": \"en\",\n \"en\": \"en-us\",\n \"en-us\": \"en-us-lessac-low\",\n \"en-gb\": \"en-gb-alan-low\",\n #\n \"spanish\": \"es\",\n \"es\": \"es-carlfm-x-low\",\n #\n \"french\": \"fr\",\n \"fr\": \"fr-siwis-low\",\n #\n \"italian\": \"it\",\n \"it\": \"it-riccardo_fasol-x-low\",\n #\n \"kazakh\": \"kk\",\n \"kk\": \"kk-iseke-x-low\",\n #\n \"nepali\": \"ne\",\n \"ne\": \"ne-google-x-low\",\n #\n \"dutch\": \"nl\",\n \"nl\": \"nl-rdh-x-low\",\n #\n \"norwegian\": \"no\",\n \"no\": \"no-talesyntese-medium\",\n #\n \"polish\": \"pl\",\n \"pl\": \"pl-mls_6892-low\",\n #\n \"portuguese\": \"pt\",\n \"pt\": \"pt-br\",\n \"pt-br\": \"pt-br-edresson-low\",\n #\n \"ukrainian\": \"uk\",\n \"uk\": \"uk-lada-x-low\",\n #\n \"vietnamese\": \"vi\",\n \"vi\": \"vi-25hours-single-low\",\n #\n \"chinese\": \"zh\",\n \"zh\": \"zh-cn\",\n \"zh-cn\": \"zh-cn-huayan-x-low\",\n}\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"name\",\n nargs=\"+\",\n choices=sorted(_VOICES.keys()),\n help=\"Voice language(s) to download\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: share)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/piper/releases/download/v0.0.2/voice-{name}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n parser.add_argument(\n \"--dry-run\", action=\"store_true\", help=\"Don't actually download\"\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if args.destination:\n args.destination = Path(args.destination)\n else:\n # Assume we're in programs/tts/piper/script\n data_dir = _DIR.parent.parent.parent.parent / \"data\"\n args.destination = data_dir / \"tts\" / \"piper\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n for voice_name in _VOICE_NAMES:\n _VOICES[voice_name] = voice_name\n\n for name in args.name:\n resolved_name = _VOICES[name]\n while resolved_name != name:\n name = resolved_name\n resolved_name = _VOICES[name]\n\n url = args.link_format.format(name=name)\n _LOGGER.info(\"Downloading %s\", url)\n\n if args.dry_run:\n return\n\n with urlopen(url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/tts/piper/script/server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nsocket_dir=\"${base_dir}/var/run\"\nmkdir -p \"${socket_dir}\"\n\npython3 \"${base_dir}/bin/piper_server.py\" --socketfile \"${socket_dir}/piper.socket\" \"$@\"\n" }, { "path": "programs/tts/piper/script/setup.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport platform\nimport shutil\nimport tarfile\nimport tempfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"setup\")\n\nPLATFORMS = {\"x86_64\": \"amd64\", \"aarch64\": \"arm64\"}\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--platform\",\n help=\"CPU architecture to download (amd64, arm64)\",\n )\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: bin)\"\n )\n parser.add_argument(\n \"--link-format\",\n default=\"https://github.com/rhasspy/piper/releases/download/v0.0.2/piper_{platform}.tar.gz\",\n help=\"Format string for download URLs\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if not args.platform:\n args.platform = platform.machine()\n\n args.platform = PLATFORMS.get(args.platform, args.platform)\n\n if not args.destination:\n args.destination = _DIR.parent / \"bin\"\n else:\n args.destination = Path(args.destination)\n\n args.destination.mkdir(parents=True, exist_ok=True)\n\n url = args.link_format.format(platform=args.platform)\n _LOGGER.info(\"Downloading %s\", url)\n with urlopen(url) as response, tempfile.TemporaryDirectory() as temp_dir_str:\n temp_dir = Path(temp_dir_str)\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", temp_dir)\n tar_gz.extractall(temp_dir)\n\n # Move piper/ contents\n piper_dir = temp_dir / \"piper\"\n for path in piper_dir.iterdir():\n rel_path = path.relative_to(piper_dir)\n if path.is_dir():\n shutil.copytree(path, args.destination / rel_path, symlinks=True)\n else:\n shutil.copy(path, args.destination / rel_path, follow_symlinks=False)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/vad/energy/bin/energy_speech_prob.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport audioop\nimport logging\nimport sys\nfrom pathlib import Path\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--threshold\",\n type=float,\n required=True,\n help=\"Energy threshold above which is considered speech\",\n )\n parser.add_argument(\n \"--width\",\n type=int,\n required=True,\n help=\"Sample width bytes\",\n )\n parser.add_argument(\n \"--samples-per-chunk\",\n required=True,\n type=int,\n help=\"Samples to send to command at a time\",\n )\n #\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n bytes_per_chunk = args.samples_per_chunk * args.width\n\n try:\n chunk = sys.stdin.buffer.read(bytes_per_chunk)\n while chunk:\n energy = get_debiased_energy(chunk, args.width)\n speech_probability = 1 if energy > args.threshold else 0\n print(speech_probability, flush=True)\n chunk = sys.stdin.buffer.read(bytes_per_chunk)\n except KeyboardInterrupt:\n pass\n\n\n# -----------------------------------------------------------------------------\n\n\ndef get_debiased_energy(audio_data: bytes, width: int) -> float:\n \"\"\"Compute RMS of debiased audio.\"\"\"\n # Thanks to the speech_recognition library!\n # https://github.com/Uberi/speech_recognition/blob/master/speech_recognition/__init__.py\n energy = -audioop.rms(audio_data, width)\n energy_bytes = bytes([energy & 0xFF, (energy >> 8) & 0xFF])\n debiased_energy = audioop.rms(\n audioop.add(audio_data, energy_bytes * (len(audio_data) // width), width),\n width,\n )\n\n return debiased_energy\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/vad/silero/README.md", "content": "# Silero VAD\n\nVoice activity detection service for Rhasspy based on [silero-vad](https://github.com/snakers4/silero-vad).\n" }, { "path": "programs/vad/silero/bin/silero_speech_prob.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport sys\nfrom dataclasses import dataclass\nfrom pathlib import Path\nfrom typing import Optional, Union\n\nimport numpy as np\nimport onnxruntime\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to Silero model\")\n parser.add_argument(\"--samples-per-chunk\", type=int, default=512)\n #\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n bytes_per_chunk = args.samples_per_chunk * 2 # 16-bit\n detector = SileroDetector(args.model)\n detector.start()\n\n try:\n chunk = sys.stdin.buffer.read(bytes_per_chunk)\n while chunk:\n speech_probability = detector.get_speech_probability(chunk)\n print(speech_probability, flush=True)\n chunk = sys.stdin.buffer.read(bytes_per_chunk)\n except (KeyboardInterrupt, BrokenPipeError):\n pass\n\n\n# -----------------------------------------------------------------------------\n\n\n@dataclass\nclass SileroDetector:\n model: Union[str, Path]\n _session: Optional[onnxruntime.InferenceSession] = None\n _h_array: Optional[np.ndarray] = None\n _c_array: Optional[np.ndarray] = None\n\n def start(self):\n _LOGGER.debug(\"Loading VAD model: %s\", self.model)\n self._session = onnxruntime.InferenceSession(str(self.model))\n self._session.intra_op_num_threads = 1\n self._session.inter_op_num_threads = 1\n\n self._h_array = np.zeros((2, 1, 64)).astype(\"float32\")\n self._c_array = np.zeros((2, 1, 64)).astype(\"float32\")\n\n def get_speech_probability(self, chunk: bytes) -> float:\n assert self._session is not None\n audio_array = np.frombuffer(chunk, dtype=np.int16)\n\n # Add batch dimension\n audio_array = np.expand_dims(audio_array, 0)\n\n ort_inputs = {\n \"input\": audio_array.astype(np.float32),\n \"h0\": self._h_array,\n \"c0\": self._c_array,\n }\n ort_outs = self._session.run(None, ort_inputs)\n out, self._h_array, self._c_array = ort_outs\n probability = out.squeeze(2)[:, 1].item()\n return probability\n\n def stop(self):\n self._session = None\n self._h_array = None\n self._c_array = None\n\n def reset(self):\n self._h_array.fill(0)\n self._c_array.fill(0)\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/vad/silero/requirements.txt", "content": "onnxruntime\nnumpy\n" }, { "path": "programs/vad/silero/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# Install rhasspy3\nrhasspy3_dir=\"${base_dir}/../../../..\"\npip3 install -e \"${rhasspy3_dir}\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/vad/silero/script/speech_prob", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/silero_speech_prob.py\" \"$@\"\n" }, { "path": "programs/vad/webrtcvad/README.md", "content": "# webrtcvad\n\nVoice activity detection service for Rhasspy based on [webrtcvad](https://pypi.org/project/webrtcvad/).\n" }, { "path": "programs/vad/webrtcvad/bin/webrtcvad_speech_prob.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport sys\nfrom pathlib import Path\n\nimport webrtcvad\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"mode\",\n choices=(0, 1, 2, 3),\n type=int,\n help=\"Aggressiveness in filtering out non-speech\",\n )\n parser.add_argument(\n \"--rate\",\n type=int,\n default=16000,\n help=\"Sample rate (hz)\",\n )\n parser.add_argument(\n \"--width\",\n type=int,\n default=2,\n help=\"Sample width bytes\",\n )\n parser.add_argument(\"--samples-per-chunk\", type=int, default=480)\n #\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n chunk_ms = 1000 * (args.samples_per_chunk / args.rate)\n assert chunk_ms in [10, 20, 30], (\n \"Sample rate and chunk size must make for 10, 20, or 30 ms buffer sizes,\"\n + f\" assuming mono audio (got {chunk_ms} ms)\"\n )\n\n bytes_per_chunk = args.samples_per_chunk * args.width\n vad = webrtcvad.Vad()\n vad.set_mode(args.mode)\n\n try:\n chunk = sys.stdin.buffer.read(bytes_per_chunk)\n while chunk:\n speech_probability = 1 if vad.is_speech(chunk, args.rate) else 0\n print(speech_probability, flush=True)\n chunk = sys.stdin.buffer.read(bytes_per_chunk)\n except KeyboardInterrupt:\n pass\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/vad/webrtcvad/requirements.txt", "content": "webrtcvad\n" }, { "path": "programs/vad/webrtcvad/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# Install rhasspy3\nrhasspy3_dir=\"${base_dir}/../../../..\"\npip3 install -e \"${rhasspy3_dir}\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/vad/webrtcvad/script/speech_prob", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/webrtcvad_speech_prob.py\" \"$@\"\n" }, { "path": "programs/wake/porcupine1/bin/list_models.py", "content": "#!/usr/bin/env python3\nfrom pathlib import Path\n\nimport pvporcupine\n\n\ndef main() -> None:\n \"\"\"Main method.\"\"\"\n\n for keyword_path in sorted(pvporcupine.pv_keyword_paths(\"\").values()):\n model_name = Path(keyword_path).name\n print(model_name)\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/wake/porcupine1/bin/porcupine_raw_text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport struct\nimport sys\nfrom pathlib import Path\n\nfrom porcupine_shared import get_arg_parser, load_porcupine\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n# -----------------------------------------------------------------------------\n\n\ndef main() -> None:\n \"\"\"Main method.\"\"\"\n parser = get_arg_parser()\n parser.add_argument(\"--samples-per-chunk\", type=int, default=512)\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n porcupine, names = load_porcupine(args)\n\n chunk_format = \"h\" * porcupine.frame_length\n bytes_per_chunk = porcupine.frame_length * 2\n\n # Read 16Khz, 16-bit mono PCM from stdin\n try:\n chunk = bytes()\n next_chunk = sys.stdin.buffer.read(bytes_per_chunk)\n while next_chunk:\n while len(chunk) >= bytes_per_chunk:\n unpacked_chunk = struct.unpack_from(\n chunk_format, chunk[:bytes_per_chunk]\n )\n keyword_index = porcupine.process(unpacked_chunk)\n if keyword_index >= 0:\n print(names[keyword_index], flush=True)\n\n chunk = chunk[bytes_per_chunk:]\n\n next_chunk = sys.stdin.buffer.read(bytes_per_chunk)\n chunk += next_chunk\n except KeyboardInterrupt:\n pass\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/wake/porcupine1/bin/porcupine_shared.py", "content": "import argparse\nfrom pathlib import Path\nfrom typing import List, Tuple\n\nimport pvporcupine\n\n\ndef get_arg_parser() -> argparse.ArgumentParser:\n \"\"\"Get shared command-line argument parser.\"\"\"\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--model\",\n required=True,\n action=\"append\",\n nargs=\"+\",\n help=\"Keyword model settings (path, [sensitivity])\",\n )\n parser.add_argument(\n \"--lang_model\",\n help=\"Path of the language model (.pv file), default is English\",\n )\n parser.add_argument(\n \"--debug\", action=\"store_true\", help=\"Print DEBUG messages to console\"\n )\n return parser\n\n\ndef load_porcupine(args: argparse.Namespace) -> Tuple[pvporcupine.Porcupine, List[str]]:\n \"\"\"Loads porcupine keywords. Returns Porcupine object and list of keyword names (in order).\"\"\"\n # Path to embedded keywords\n keyword_dir = Path(next(iter(pvporcupine.pv_keyword_paths(\"\").values()))).parent\n\n names: List[str] = []\n keyword_paths: List[Path] = []\n sensitivities: List[float] = []\n\n model_path = (\n str(Path(args.lang_model).absolute()) if args.lang_model is not None else None\n )\n\n for model_settings in args.model:\n keyword_path_str = model_settings[0]\n keyword_path = Path(keyword_path_str)\n if not keyword_path.exists():\n keyword_path = keyword_dir / keyword_path_str\n assert keyword_path.exists(), f\"Cannot find {keyword_path_str}\"\n\n keyword_paths.append(keyword_path)\n names.append(keyword_path.stem)\n\n sensitivity = 0.5\n if len(model_settings) > 1:\n sensitivity = float(model_settings[1])\n\n sensitivities.append(sensitivity)\n\n porcupine = pvporcupine.create(\n keyword_paths=[str(keyword_path.absolute()) for keyword_path in keyword_paths],\n sensitivities=sensitivities,\n model_path=model_path,\n )\n\n return porcupine, names\n" }, { "path": "programs/wake/porcupine1/bin/porcupine_stream.py", "content": "#!/usr/bin/env python3\nimport logging\nimport struct\nfrom pathlib import Path\n\nfrom porcupine_shared import get_arg_parser, load_porcupine\n\nfrom rhasspy3.audio import AudioChunk, AudioStop\nfrom rhasspy3.event import read_event, write_event\nfrom rhasspy3.wake import Detection, NotDetected\n\n_FILE = Path(__file__)\n_DIR = _FILE.parent\n_LOGGER = logging.getLogger(_FILE.stem)\n\n# -----------------------------------------------------------------------------\n\n\ndef main() -> None:\n \"\"\"Main method.\"\"\"\n parser = get_arg_parser()\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n porcupine, names = load_porcupine(args)\n\n chunk_format = \"h\" * porcupine.frame_length\n bytes_per_chunk = porcupine.frame_length * 2 # 16-bit width\n audio_bytes = bytes()\n is_detected = False\n\n try:\n while True:\n event = read_event()\n if event is None:\n break\n\n if AudioStop.is_type(event.type):\n break\n\n if not AudioChunk.is_type(event.type):\n continue\n\n chunk = AudioChunk.from_event(event)\n audio_bytes += chunk.audio\n\n while len(audio_bytes) >= bytes_per_chunk:\n unpacked_chunk = struct.unpack_from(\n chunk_format, audio_bytes[:bytes_per_chunk]\n )\n keyword_index = porcupine.process(unpacked_chunk)\n if keyword_index >= 0:\n write_event(\n Detection(\n name=names[keyword_index], timestamp=chunk.timestamp\n ).event()\n )\n is_detected = True\n\n audio_bytes = audio_bytes[bytes_per_chunk:]\n\n if is_detected:\n write_event(NotDetected().event())\n except KeyboardInterrupt:\n pass\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/wake/porcupine1/requirements.txt", "content": "pvporcupine~=1.9.0\n" }, { "path": "programs/wake/porcupine1/script/download.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport tarfile\nfrom pathlib import Path\nfrom urllib.request import urlopen\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"download\")\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--destination\", help=\"Path to destination directory (default: data)\"\n )\n parser.add_argument(\n \"--url\",\n default=\"https://github.com/rhasspy/models/releases/download/v1.0/wake_porcupine1-data.tar.gz\",\n help=\"URL of porcupine1 data\",\n )\n args = parser.parse_args()\n logging.basicConfig(level=logging.INFO)\n\n if args.destination:\n args.destination = Path(args.destination)\n else:\n # Assume we're in programs/wake/porcupine1/script\n data_dir = _DIR.parent.parent.parent.parent / \"data\"\n args.destination = data_dir / \"wake\" / \"porcupine1\"\n\n args.destination.parent.mkdir(parents=True, exist_ok=True)\n\n _LOGGER.info(\"Downloading %s\", args.url)\n with urlopen(args.url) as response:\n with tarfile.open(mode=\"r|*\", fileobj=response) as tar_gz:\n _LOGGER.info(\"Extracting to %s\", args.destination)\n tar_gz.extractall(args.destination)\n\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/wake/porcupine1/script/list_models", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/list_models.py\" \"$@\"\n" }, { "path": "programs/wake/porcupine1/script/raw2text", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\npython3 \"${base_dir}/bin/porcupine_raw_text.py\" \"$@\"\n" }, { "path": "programs/wake/porcupine1/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/wake/precise-lite/bin/precise.py", "content": "#!/usr/bin/env python3\n# Copyright 2021 Mycroft AI Inc.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport argparse\nimport logging\nimport os\nimport sys\nimport typing\nfrom dataclasses import dataclass\nfrom enum import IntEnum\nfrom math import floor\nfrom pathlib import Path\nfrom typing import Any, Optional, Union\n\nimport numpy as np\nimport tflite_runtime.interpreter as tflite\nfrom sonopy import mfcc_spec\n\nMAX_WAV_VALUE = 32768\n_log = logging.getLogger(\"mycroft_hotword\")\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\"model\", help=\"Path to TFLite model\")\n parser.add_argument(\n \"--sensitivity\",\n type=float,\n default=0.8,\n help=\"Model sensitivity (0-1, default: 0.8)\",\n )\n parser.add_argument(\n \"--trigger-level\",\n type=int,\n default=4,\n help=\"Number of activations before detection occurs (default: 4)\",\n )\n parser.add_argument(\n \"--chunk-size\",\n type=int,\n default=2048,\n help=\"Number of bytes to read at a time from stdin\",\n )\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n args.model = Path(args.model).absolute()\n engine = TFLiteHotWordEngine(\n local_model_file=args.model,\n sensitivity=args.sensitivity,\n trigger_level=args.trigger_level,\n chunk_size=args.chunk_size,\n )\n\n if os.isatty(sys.stdin.fileno()):\n print(\"Reading raw 16-bit 16khz mono audio from stdin\", file=sys.stderr)\n\n is_first_audio = True\n try:\n while True:\n chunk = sys.stdin.buffer.read(args.chunk_size)\n if not chunk:\n break\n\n if is_first_audio:\n _log.info(\"Receiving audio\")\n is_first_audio = False\n\n engine.update(chunk)\n\n if engine.found_wake_word(None):\n print(args.model.name, flush=True)\n except KeyboardInterrupt:\n pass\n\n\n# -----------------------------------------------------------------------------\n\n\nclass TFLiteHotWordEngine:\n def __init__(\n self,\n local_model_file: Union[str, Path],\n sensitivity: float = 0.7,\n trigger_level: int = 4,\n chunk_size: int = 2048,\n ):\n self.sensitivity = sensitivity\n self.trigger_level = trigger_level\n self.chunk_size = chunk_size\n\n self.model_path = Path(local_model_file).absolute()\n\n self._interpreter: Optional[tflite.Interpreter] = None\n self._params: Optional[ListenerParams] = None\n self._input_details: Optional[Any] = None\n self._output_details: Optional[Any] = None\n\n # Rolling window of MFCCs (fixed sized)\n self._inputs: Optional[np.ndarray] = None\n\n # Current MFCC timestep\n self._inputs_idx: int = 0\n\n # Bytes for one window of audio\n self._window_bytes: int = 0\n\n # Bytes for one MFCC hop\n self._hop_bytes: int = 0\n\n # Raw audio\n self._chunk_buffer = bytes()\n\n # Activation level (> trigger_level = wake word found)\n self._activation: int = 0\n\n # True if wake word was found during last update\n self._is_found = False\n\n # There doesn't seem to be an initialize() method for wake word plugins,\n # so we'll load the model here.\n self._load_model()\n\n # Last probability\n self._probability: Optional[float] = None\n\n def _load_model(self):\n _log.debug(\"Loading model from %s\", self.model_path)\n self._interpreter = tflite.Interpreter(model_path=str(self.model_path))\n self._interpreter.allocate_tensors()\n self._input_details = self._interpreter.get_input_details()\n self._output_details = self._interpreter.get_output_details()\n\n # TODO: Load these from adjacent file\n self._params = ListenerParams()\n\n self._window_bytes = self._params.window_samples * self._params.sample_depth\n self._hop_bytes = self._params.hop_samples * self._params.sample_depth\n\n # Rolling window of MFCCs (fixed sized)\n self._inputs = np.zeros(\n (1, self._params.n_features, self._params.n_mfcc), dtype=np.float32\n )\n\n def update(self, chunk):\n self._is_found = False\n self._chunk_buffer += chunk\n self._probability = None\n\n # Process all available windows\n while len(self._chunk_buffer) >= self._window_bytes:\n # Process current audio\n audio = buffer_to_audio(self._chunk_buffer)\n\n # TODO: Implement different MFCC algorithms\n mfccs = mfcc_spec(\n audio,\n self._params.sample_rate,\n (self._params.window_samples, self._params.hop_samples),\n num_filt=self._params.n_filt,\n fft_size=self._params.n_fft,\n num_coeffs=self._params.n_mfcc,\n )\n\n num_timesteps = mfccs.shape[0]\n\n # Remove processed audio from buffer\n self._chunk_buffer = self._chunk_buffer[num_timesteps * self._hop_bytes :]\n\n # Check if we have a full set of inputs yet\n inputs_end_idx = self._inputs_idx + num_timesteps\n if inputs_end_idx > self._inputs.shape[1]:\n # Full set, need to roll back existing inputs\n self._inputs = np.roll(self._inputs, -num_timesteps, axis=1)\n inputs_end_idx = self._inputs.shape[1]\n self._inputs_idx = inputs_end_idx - num_timesteps\n\n # Insert new MFCCs at the end\n self._inputs[0, self._inputs_idx : inputs_end_idx, :] = mfccs\n self._inputs_idx += num_timesteps\n if inputs_end_idx < self._inputs.shape[1]:\n # Don't have a full set of inputs yet\n continue\n\n # TODO: Add deltas\n\n # raw_output\n self._interpreter.set_tensor(self._input_details[0][\"index\"], self._inputs)\n self._interpreter.invoke()\n raw_output = self._interpreter.get_tensor(self._output_details[0][\"index\"])\n prob = raw_output[0][0]\n\n if (prob < 0.0) or (prob > 1.0):\n # TODO: Handle out of range.\n # Not seeing these currently, so ignoring.\n continue\n\n self._probability = prob.item()\n\n # Decode\n activated = prob > 1.0 - self.sensitivity\n triggered = False\n if activated or (self._activation < 0):\n # Increase activation\n self._activation += 1\n\n triggered = self._activation > self.trigger_level\n if triggered or (activated and (self._activation < 0)):\n # Push activation down far to avoid an accidental re-activation\n self._activation = -(8 * 2048) // self.chunk_size\n elif self._activation > 0:\n # Decrease activation\n self._activation -= 1\n\n if triggered:\n self._is_found = True\n _log.debug(\"Triggered\")\n break\n\n return self._is_found\n\n def found_wake_word(self, frame_data):\n return self._is_found\n\n def reset(self):\n self._inputs = np.zeros(\n (1, self._params.n_features, self._params.n_mfcc), dtype=np.float32\n )\n self._activation = 0\n self._is_found = False\n self._inputs_idx = 0\n self._chunk_buffer = bytes()\n\n @property\n def probability(self) -> Optional[float]:\n return self._probability\n\n\n# -----------------------------------------------------------------------------\n\n\nclass Vectorizer(IntEnum):\n \"\"\"\n Chooses which function to call to vectorize audio\n\n Options:\n mels: Convert to a compressed Mel spectrogram\n mfccs: Convert to a MFCC spectrogram\n speechpy_mfccs: Legacy option to convert to MFCCs using old library\n \"\"\"\n\n mels = 1\n mfccs = 2\n speechpy_mfccs = 3\n\n\n@dataclass\nclass ListenerParams:\n \"\"\"\n General pipeline information:\n - Audio goes through a series of transformations to convert raw audio into machine readable data\n - These transformations are as follows:\n - Raw audio -> chopped audio\n - buffer_t, sample_depth: Input audio loaded and truncated using these value\n - window_t, hop_t: Linear audio chopped into overlapping frames using a sliding window\n - Chopped audio -> FFT spectrogram\n - n_fft, sample_rate: Each audio frame is converted to n_fft frequency intensities\n - FFT spectrogram -> Mel spectrogram (compressed)\n - n_filt: Each fft frame is compressed to n_filt summarized mel frequency bins/bands\n - Mel spectrogram -> MFCC\n - n_mfcc: Each mel frame is converted to MFCCs and the first n_mfcc values are taken\n - Disabled by default: Last phase -> Delta vectors\n - use_delta: If this value is true, the difference between consecutive vectors is concatenated to each frame\n\n Parameters for audio pipeline:\n - buffer_t: Input size of audio. Wakeword must fit within this time\n - window_t: Time of the window used to calculate a single spectrogram frame\n - hop_t: Time the window advances forward to calculate the next spectrogram frame\n - sample_rate: Input audio sample rate\n - sample_depth: Bytes per input audio sample\n - n_fft: Size of FFT to generate from audio frame\n - n_filt: Number of filters to compress FFT to\n - n_mfcc: Number of MFCC coefficients to use\n - use_delta: If True, generates \"delta vectors\" before sending to network\n - vectorizer: The type of input fed into the network. Options listed in class Vectorizer\n - threshold_config: Output distribution configuration automatically generated from precise-calc-threshold\n - threshold_center: Output distribution center automatically generated from precise-calc-threshold\n \"\"\"\n\n buffer_t: float = 1.5\n window_t: float = 0.1\n hop_t: float = 0.05\n sample_rate: int = 16000\n sample_depth: int = 2\n n_fft: int = 512\n n_filt: int = 20\n n_mfcc: int = 13\n use_delta: bool = False\n vectorizer: int = Vectorizer.mfccs\n threshold_config: typing.Tuple[typing.Tuple[int, ...], ...] = ((6, 4),)\n threshold_center: float = 0.2\n\n @property\n def buffer_samples(self):\n \"\"\"buffer_t converted to samples, truncating partial frames\"\"\"\n samples = int(self.sample_rate * self.buffer_t + 0.5)\n return self.hop_samples * (samples // self.hop_samples)\n\n @property\n def n_features(self):\n \"\"\"Number of timesteps in one input to the network\"\"\"\n return 1 + int(\n floor((self.buffer_samples - self.window_samples) / self.hop_samples)\n )\n\n @property\n def window_samples(self):\n \"\"\"window_t converted to samples\"\"\"\n return int(self.sample_rate * self.window_t + 0.5)\n\n @property\n def hop_samples(self):\n \"\"\"hop_t converted to samples\"\"\"\n return int(self.sample_rate * self.hop_t + 0.5)\n\n @property\n def max_samples(self):\n \"\"\"The input size converted to audio samples\"\"\"\n return int(self.buffer_t * self.sample_rate)\n\n @property\n def feature_size(self):\n \"\"\"The size of an input vector generated with these parameters\"\"\"\n num_features = {\n Vectorizer.mfccs: self.n_mfcc,\n Vectorizer.mels: self.n_filt,\n Vectorizer.speechpy_mfccs: self.n_mfcc,\n }[self.vectorizer]\n if self.use_delta:\n num_features *= 2\n return num_features\n\n\ndef chunk_audio(\n audio: np.ndarray, chunk_size: int\n) -> typing.Generator[np.ndarray, None, None]:\n for i in range(chunk_size, len(audio), chunk_size):\n yield audio[i - chunk_size : i]\n\n\ndef buffer_to_audio(audio_buffer: bytes) -> np.ndarray:\n \"\"\"Convert a raw mono audio byte string to numpy array of floats\"\"\"\n return np.frombuffer(audio_buffer, dtype=\" bytes:\n \"\"\"Convert a numpy array of floats to raw mono audio\"\"\"\n return (audio * MAX_WAV_VALUE).astype(\"=2.5.0,<3.0\n" }, { "path": "programs/wake/precise-lite/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "programs/wake/snowboy/bin/snowboy_raw_text.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport logging\nimport sys\nfrom pathlib import Path\nfrom typing import Dict\n\nfrom snowboy import snowboydecoder, snowboydetect\n\n_LOGGER = logging.getLogger(\"snowboy_raw_text\")\n\n# -----------------------------------------------------------------------------\n\n\ndef main() -> None:\n \"\"\"Main method.\"\"\"\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"--model\",\n required=True,\n action=\"append\",\n nargs=\"+\",\n help=\"Snowboy model settings (path, [sensitivity], [audio_gain], [apply_frontend])\",\n )\n parser.add_argument(\"--samples-per-chunk\", type=int, default=1024)\n args = parser.parse_args()\n\n # logging.basicConfig wouldn't work if a handler already existed.\n # snowboy must mess with logging, so this resets it.\n logging.getLogger().handlers = []\n logging.basicConfig(level=logging.INFO)\n\n # Load model settings\n detectors: Dict[str, snowboydetect.SnowboyDetect] = {}\n\n for model_settings in args.model:\n model_path = Path(model_settings[0])\n\n sensitivity = \"0.5\"\n if len(model_settings) > 1:\n sensitivity = model_settings[1]\n\n audio_gain = 1.0\n if len(model_settings) > 2:\n audio_gain = float(model_settings[2])\n\n apply_frontend = False\n if len(model_settings) > 3:\n apply_frontend = model_settings[3].strip().lower() == \"true\"\n\n detector = snowboydetect.SnowboyDetect(\n snowboydecoder.RESOURCE_FILE.encode(), str(model_path).encode()\n )\n\n detector.SetSensitivity(sensitivity.encode())\n detector.SetAudioGain(audio_gain)\n detector.ApplyFrontend(apply_frontend)\n\n detectors[model_path.stem] = detector\n\n # Read 16Khz, 16-bit mono PCM from stdin\n bytes_per_chunk = args.samples_per_chunk * 2\n try:\n chunk = bytes()\n next_chunk = sys.stdin.buffer.read(bytes_per_chunk)\n while next_chunk:\n while len(chunk) >= bytes_per_chunk:\n for name, detector in detectors.items():\n # Return is:\n # -2 silence\n # -1 error\n # 0 voice\n # n index n-1\n result_index = detector.RunDetection(chunk[:bytes_per_chunk])\n\n if result_index > 0:\n # Detection\n print(name, flush=True)\n\n chunk = chunk[bytes_per_chunk:]\n\n next_chunk = sys.stdin.buffer.read(args.samples_per_chunk)\n chunk += next_chunk\n except KeyboardInterrupt:\n pass\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "programs/wake/snowboy/requirements.txt", "content": "snowboy @ https://github.com/Kitt-AI/snowboy/archive/v1.3.0.tar.gz\n" }, { "path": "programs/wake/snowboy/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\nif [ ! -d \"${venv}\" ]; then\n # Create virtual environment\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "pylintrc", "content": "[MESSAGES CONTROL]\ndisable=\n format,\n abstract-method,\n cyclic-import,\n duplicate-code,\n global-statement,\n import-outside-toplevel,\n inconsistent-return-statements,\n locally-disabled,\n not-context-manager,\n too-few-public-methods,\n too-many-arguments,\n too-many-branches,\n too-many-instance-attributes,\n too-many-lines,\n too-many-locals,\n too-many-public-methods,\n too-many-return-statements,\n too-many-statements,\n too-many-boolean-expressions,\n unnecessary-pass,\n unused-argument,\n broad-except,\n too-many-nested-blocks,\n invalid-name,\n unused-import,\n fixme,\n useless-super-delegation,\n missing-module-docstring,\n missing-class-docstring,\n missing-function-docstring,\n import-error,\n consider-using-with\n\n[FORMAT]\nexpected-line-ending-format=LF\n" }, { "path": "requirements_dev.txt", "content": "black==22.12.0\nflake8==6.0.0\nisort==5.11.3\nmypy==0.991\npylint==2.15.9\npytest==7.2.0\n" }, { "path": "requirements_http_api.txt", "content": "quart\nQuart-CORS\nhypercorn\n" }, { "path": "rhasspy3/VERSION", "content": "0.0.1\n" }, { "path": "rhasspy3/__init__.py", "content": "" }, { "path": "rhasspy3/asr.py", "content": "\"\"\"Speech to text.\"\"\"\nimport asyncio\nimport logging\nimport wave\nfrom dataclasses import dataclass\nfrom typing import IO, AsyncIterable, Optional, Union\n\nfrom .audio import AudioChunk, AudioStart, AudioStop, wav_to_chunks\nfrom .config import PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event, async_write_event\nfrom .program import create_process\nfrom .vad import DOMAIN as VAD_DOMAIN\nfrom .vad import VoiceStarted, VoiceStopped\n\nDOMAIN = \"asr\"\n_TRANSCRIPT_TYPE = \"transcript\"\n\n_LOGGER = logging.getLogger(__name__)\n\n\n@dataclass\nclass Transcript(Eventable):\n text: str\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _TRANSCRIPT_TYPE\n\n def event(self) -> Event:\n return Event(type=_TRANSCRIPT_TYPE, data={\"text\": self.text})\n\n @staticmethod\n def from_event(event: Event) -> \"Transcript\":\n assert event.data is not None\n return Transcript(text=event.data[\"text\"])\n\n\nasync def transcribe(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n wav_in: IO[bytes],\n samples_per_chunk: int,\n) -> Optional[Transcript]:\n transcript: Optional[Transcript] = None\n wav_file: wave.Wave_read = wave.open(wav_in, \"rb\")\n with wav_file:\n rate = wav_file.getframerate()\n width = wav_file.getsampwidth()\n channels = wav_file.getnchannels()\n\n async with (await create_process(rhasspy, DOMAIN, program)) as asr_proc:\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n\n timestamp = 0\n await async_write_event(\n AudioStart(rate, width, channels, timestamp=timestamp).event(),\n asr_proc.stdin,\n )\n\n is_first_chunk = True\n\n for chunk in wav_to_chunks(wav_file, samples_per_chunk=samples_per_chunk):\n if is_first_chunk:\n is_first_chunk = False\n _LOGGER.debug(\"transcribe: processing audio\")\n\n await async_write_event(chunk.event(), asr_proc.stdin)\n if chunk.timestamp is not None:\n timestamp = chunk.timestamp\n else:\n timestamp += chunk.milliseconds\n\n await async_write_event(\n AudioStop(timestamp=timestamp).event(), asr_proc.stdin\n )\n\n _LOGGER.debug(\"transcribe: audio finished\")\n\n while True:\n event = await async_read_event(asr_proc.stdout)\n if event is None:\n break\n\n if Transcript.is_type(event.type):\n transcript = Transcript.from_event(event)\n _LOGGER.debug(\"transcribe: %s\", transcript)\n break\n\n return transcript\n\n\nasync def transcribe_stream(\n rhasspy: Rhasspy,\n asr_program: Union[str, PipelineProgramConfig],\n vad_program: Union[str, PipelineProgramConfig],\n audio_stream: AsyncIterable[bytes],\n rate: int,\n width: int,\n channels: int,\n) -> Optional[Transcript]:\n transcript: Optional[Transcript] = None\n async with (await create_process(rhasspy, DOMAIN, asr_program)) as asr_proc, (\n await create_process(rhasspy, VAD_DOMAIN, vad_program)\n ) as vad_proc:\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n assert vad_proc.stdin is not None\n assert vad_proc.stdout is not None\n\n timestamp = 0\n audio_start_event = AudioStart(\n rate, width, channels, timestamp=timestamp\n ).event()\n await asyncio.gather(\n async_write_event(\n audio_start_event,\n asr_proc.stdin,\n ),\n async_write_event(\n audio_start_event,\n vad_proc.stdin,\n ),\n )\n\n async def next_chunk():\n \"\"\"Get the next chunk from audio stream.\"\"\"\n async for chunk_bytes in audio_stream:\n return chunk_bytes\n\n is_first_chunk = True\n audio_task = asyncio.create_task(next_chunk())\n vad_task = asyncio.create_task(async_read_event(vad_proc.stdout))\n pending = {audio_task, vad_task}\n\n while True:\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n\n if vad_task in done:\n vad_event = vad_task.result()\n if vad_event is None:\n break\n\n if VoiceStarted.is_type(vad_event.type):\n _LOGGER.debug(\"transcribe: voice started\")\n elif VoiceStopped.is_type(vad_event.type):\n _LOGGER.debug(\"transcribe: voice stopped\")\n break\n\n vad_task = asyncio.create_task(async_read_event(vad_proc.stdout))\n pending.add(vad_task)\n\n if audio_task in done:\n chunk_bytes = audio_task.result()\n if not chunk_bytes:\n # End of audio stream\n break\n\n if is_first_chunk:\n _LOGGER.debug(\"transcribe: processing audio\")\n is_first_chunk = False\n\n chunk = AudioChunk(rate, width, channels, chunk_bytes)\n chunk_event = chunk.event()\n await asyncio.gather(\n async_write_event(chunk_event, asr_proc.stdin),\n async_write_event(chunk_event, vad_proc.stdin),\n )\n timestamp += chunk.milliseconds\n\n audio_task = asyncio.create_task(next_chunk())\n pending.add(audio_task)\n\n await async_write_event(AudioStop(timestamp=timestamp).event(), asr_proc.stdin)\n _LOGGER.debug(\"transcribe: audio finished\")\n\n while True:\n event = await async_read_event(asr_proc.stdout)\n if event is None:\n break\n\n if Transcript.is_type(event.type):\n transcript = Transcript.from_event(event)\n _LOGGER.debug(\"transcribe: %s\", transcript)\n break\n\n return transcript\n" }, { "path": "rhasspy3/audio.py", "content": "\"\"\"Audio input/output.\"\"\"\nimport audioop\nimport wave\nfrom dataclasses import dataclass\nfrom typing import Iterable, Optional\n\nfrom .event import Event, Eventable\n\n_TYPE = \"audio-chunk\"\n_START_TYPE = \"audio-start\"\n_STOP_TYPE = \"audio-stop\"\n\nDEFAULT_IN_RATE = 16000 # Hz\nDEFAULT_OUT_RATE = 22050 # Hz\n\nDEFAULT_IN_WIDTH = 2 # bytes\nDEFAULT_OUT_WIDTH = 2 # bytes\n\nDEFAULT_IN_CHANNELS = 1 # mono\nDEFAULT_OUT_CHANNELS = 1 # mono\n\nDEFAULT_SAMPLES_PER_CHUNK = 1024\n\n\n@dataclass\nclass AudioChunk(Eventable):\n \"\"\"Chunk of raw PCM audio.\"\"\"\n\n rate: int\n \"\"\"Hertz\"\"\"\n\n width: int\n \"\"\"Bytes\"\"\"\n\n channels: int\n \"\"\"Mono = 1\"\"\"\n\n audio: bytes\n \"\"\"Raw audio\"\"\"\n\n timestamp: Optional[int] = None\n \"\"\"Milliseconds\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _TYPE\n\n def event(self) -> Event:\n return Event(\n type=_TYPE,\n data={\n \"rate\": self.rate,\n \"width\": self.width,\n \"channels\": self.channels,\n \"timestamp\": self.timestamp,\n },\n payload=self.audio,\n )\n\n @staticmethod\n def from_event(event: Event) -> \"AudioChunk\":\n assert event.data is not None\n assert event.payload is not None\n\n return AudioChunk(\n rate=event.data[\"rate\"],\n width=event.data[\"width\"],\n channels=event.data[\"channels\"],\n audio=event.payload,\n timestamp=event.data.get(\"timestamp\"),\n )\n\n @property\n def samples(self) -> int:\n return len(self.audio) // (self.width * self.channels)\n\n @property\n def seconds(self) -> float:\n return self.samples / self.rate\n\n @property\n def milliseconds(self) -> int:\n return int(self.seconds * 1_000)\n\n\n@dataclass\nclass AudioStart(Eventable):\n \"\"\"Audio stream has started.\"\"\"\n\n rate: int\n \"\"\"Hertz\"\"\"\n\n width: int\n \"\"\"Bytes\"\"\"\n\n channels: int\n \"\"\"Mono = 1\"\"\"\n\n timestamp: Optional[int] = None\n \"\"\"Milliseconds\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _START_TYPE\n\n def event(self) -> Event:\n\n return Event(\n type=_START_TYPE,\n data={\n \"rate\": self.rate,\n \"width\": self.width,\n \"channels\": self.channels,\n \"timestamp\": self.timestamp,\n },\n )\n\n @staticmethod\n def from_event(event: Event) -> \"AudioStart\":\n assert event.data is not None\n return AudioStart(\n rate=event.data[\"rate\"],\n width=event.data[\"width\"],\n channels=event.data[\"channels\"],\n timestamp=event.data.get(\"timestamp\"),\n )\n\n\n@dataclass\nclass AudioStop(Eventable):\n \"\"\"Audio stream has stopped.\"\"\"\n\n timestamp: Optional[int] = None\n \"\"\"Milliseconds\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _STOP_TYPE\n\n def event(self) -> Event:\n return Event(\n type=_STOP_TYPE,\n data={\"timestamp\": self.timestamp},\n )\n\n @staticmethod\n def from_event(event: Event) -> \"AudioStop\":\n return AudioStop(timestamp=event.data.get(\"timestamp\"))\n\n\n@dataclass\nclass AudioChunkConverter:\n \"\"\"Converts audio chunks using audioop.\"\"\"\n\n rate: Optional[int] = None\n width: Optional[int] = None\n channels: Optional[int] = None\n _ratecv_state = None\n\n def convert(self, chunk: AudioChunk) -> AudioChunk:\n \"\"\"Converts sample rate, width, and channels as necessary.\"\"\"\n if (\n ((self.rate is None) or (chunk.rate == self.rate))\n and ((self.width is None) or (chunk.width == self.width))\n and ((self.channels is None) or (chunk.channels == self.channels))\n ):\n return chunk\n\n audio_bytes = chunk.audio\n width = chunk.width\n\n if (self.width is not None) and (chunk.width != self.width):\n # Convert sample width\n audio_bytes = audioop.lin2lin(audio_bytes, chunk.width, self.width)\n width = self.width\n\n channels = chunk.channels\n if (self.channels is not None) and (chunk.channels != self.channels):\n # Convert to mono or stereo\n if self.channels == 1:\n audio_bytes = audioop.tomono(audio_bytes, width, 1.0, 1.0)\n elif self.channels == 2:\n audio_bytes = audioop.tostereo(audio_bytes, width, 1.0, 1.0)\n else:\n raise ValueError(f\"Cannot convert to channels: {self.channels}\")\n\n channels = self.channels\n\n rate = chunk.rate\n if (self.rate is not None) and (chunk.rate != self.rate):\n # Resample\n audio_bytes, self._ratecv_state = audioop.ratecv(\n audio_bytes,\n width,\n channels,\n chunk.rate,\n self.rate,\n self._ratecv_state,\n )\n rate = self.rate\n\n return AudioChunk(rate, width, channels, audio_bytes, timestamp=chunk.timestamp)\n\n\ndef wav_to_chunks(\n wav_file: wave.Wave_read, samples_per_chunk: int, timestamp: int = 0\n) -> Iterable[AudioChunk]:\n \"\"\"Splits WAV file into AudioChunks.\"\"\"\n rate = wav_file.getframerate()\n width = wav_file.getsampwidth()\n channels = wav_file.getnchannels()\n audio_bytes = wav_file.readframes(samples_per_chunk)\n while audio_bytes:\n chunk = AudioChunk(\n rate=rate,\n width=width,\n channels=channels,\n audio=audio_bytes,\n timestamp=timestamp,\n )\n yield chunk\n timestamp += chunk.milliseconds\n audio_bytes = wav_file.readframes(samples_per_chunk)\n" }, { "path": "rhasspy3/config.py", "content": "import argparse\nfrom dataclasses import dataclass, field\nfrom typing import Any, Dict, Optional\n\nfrom .util import merge_dict\nfrom .util.dataclasses_json import DataClassJsonMixin\nfrom .util.jaml import safe_load\n\n\n@dataclass\nclass CommandConfig(DataClassJsonMixin):\n command: str\n shell: bool = False\n\n\n@dataclass\nclass ProgramDownloadConfig(DataClassJsonMixin):\n description: Optional[str] = None\n check_file: Optional[str] = None\n\n\n@dataclass\nclass ProgramInstallConfig(CommandConfig):\n check_file: Optional[str] = None\n download: Optional[CommandConfig] = None\n downloads: Optional[Dict[str, ProgramDownloadConfig]] = None\n\n\n@dataclass\nclass ProgramConfig(CommandConfig):\n adapter: Optional[str] = None\n template_args: Optional[Dict[str, Any]] = None\n installed: bool = True\n install: Optional[ProgramInstallConfig] = None\n\n\n@dataclass\nclass PipelineProgramConfig(DataClassJsonMixin):\n name: str\n template_args: Optional[Dict[str, Any]] = None\n after: Optional[CommandConfig] = None\n\n\n@dataclass\nclass PipelineConfig(DataClassJsonMixin):\n inherit: Optional[str] = None\n mic: Optional[PipelineProgramConfig] = None\n wake: Optional[PipelineProgramConfig] = None\n vad: Optional[PipelineProgramConfig] = None\n asr: Optional[PipelineProgramConfig] = None\n intent: Optional[PipelineProgramConfig] = None\n handle: Optional[PipelineProgramConfig] = None\n tts: Optional[PipelineProgramConfig] = None\n snd: Optional[PipelineProgramConfig] = None\n\n\n@dataclass\nclass SatelliteConfig(DataClassJsonMixin):\n mic: Optional[PipelineProgramConfig] = None\n wake: Optional[PipelineProgramConfig] = None\n remote: Optional[PipelineProgramConfig] = None\n snd: Optional[PipelineProgramConfig] = None\n\n\n@dataclass\nclass ServerConfig(DataClassJsonMixin):\n command: str\n shell: bool = False\n template_args: Optional[Dict[str, Any]] = None\n\n\n@dataclass\nclass Config(DataClassJsonMixin):\n programs: Dict[str, Dict[str, ProgramConfig]]\n \"\"\"domain -> name -> program\"\"\"\n\n pipelines: Dict[str, PipelineConfig] = field(default_factory=dict)\n \"\"\"name -> pipeline\"\"\"\n\n satellites: Dict[str, SatelliteConfig] = field(default_factory=dict)\n \"\"\"name -> satellite\"\"\"\n\n servers: Dict[str, Dict[str, ServerConfig]] = field(default_factory=dict)\n \"\"\"domain -> name -> server\"\"\"\n\n def __post_init__(self):\n # Handle inheritance\n # TODO: Catch loops\n pipeline_queue = list(self.pipelines.values())\n while pipeline_queue:\n child_pipeline = pipeline_queue.pop()\n if child_pipeline.inherit:\n parent_pipeline = self.pipelines[child_pipeline.inherit]\n if parent_pipeline.inherit:\n # Need to process parent first\n pipeline_queue.append(child_pipeline)\n continue\n\n child_pipeline.mic = child_pipeline.mic or parent_pipeline.mic\n child_pipeline.wake = child_pipeline.wake or parent_pipeline.wake\n child_pipeline.vad = child_pipeline.vad or parent_pipeline.vad\n child_pipeline.asr = child_pipeline.asr or parent_pipeline.asr\n child_pipeline.intent = child_pipeline.intent or parent_pipeline.intent\n child_pipeline.handle = child_pipeline.handle or parent_pipeline.handle\n child_pipeline.tts = child_pipeline.tts or parent_pipeline.tts\n child_pipeline.snd = child_pipeline.snd or parent_pipeline.snd\n\n # Mark as done\n child_pipeline.inherit = None\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser()\n parser.add_argument(\"config\", nargs=\"+\", help=\"Path to YAML configuration file\")\n args = parser.parse_args()\n\n config_dict: Dict[str, Any] = {}\n for config_path in args.config:\n with open(config_path, \"r\", encoding=\"utf-8\") as config_file:\n merge_dict(config_dict, safe_load(config_file))\n\n config = Config.from_dict(config_dict)\n print(config)\n" }, { "path": "rhasspy3/configuration.yaml", "content": "programs:\n\n # -----------\n # Audio input\n # -----------\n mic:\n\n # apt-get install alsa-utils\n arecord:\n command: |\n arecord -q -D \"${device}\" -r 16000 -c 1 -f S16_LE -t raw -\n adapter: |\n mic_adapter_raw.py --samples-per-chunk 1024 --rate 16000 --width 2 --channels 1\n template_args:\n device: \"default\"\n\n # https://people.csail.mit.edu/hubert/pyaudio/docs/\n pyaudio:\n command: |\n script/events\n\n # https://python-sounddevice.readthedocs.io\n sounddevice:\n command: |\n script/events\n\n # apt-get install gstreamer1.0-tools gstreamer1.0-plugins-base\n gstreamer_udp:\n command: |\n gst-launch-1.0 -v udpsrc address=${address} port=${port} ! rawaudioparse use-sink-caps=false format=pcm pcm-format=${format} sample-rate=${rate} num-channels=${channels} ! audioconvert ! audioresample ! volume volume=3.0 ! level ! fdsink fd=1 sync=false\n template_args:\n format: s16le\n rate: 16000\n channels: 1\n address: \"0.0.0.0\"\n port: 5000\n adapter: |\n mic_adapter_raw.py --samples-per-chunk 1024 --rate 16000 --width 2 --channels 1\n\n udp_raw:\n command: |\n bin/udp_raw.py --host ${host} --port ${port}\n template_args:\n host: 0.0.0.0\n port: 5000\n\n # -------------------\n # Wake word detection\n # -------------------\n wake:\n\n # https://github.com/Picovoice/porcupine\n # Models: see script/list_models\n porcupine1:\n command: |\n .venv/bin/python3 bin/porcupine_stream.py --model \"${model}\"\n template_args:\n model: \"porcupine_linux.ppn\"\n\n # https://github.com/Kitt-AI/snowboy\n # Models included in share/\n # Custom wake word: https://github.com/rhasspy/snowboy-seasalt\n snowboy:\n command: |\n .venv/bin/python3 bin/snowboy_raw_text.py --model \"${model}\"\n adapter: |\n wake_adapter_raw.py\n template_args:\n model: \"share/snowboy.umdl\"\n\n # https://github.com/mycroftAI/mycroft-precise\n # Model included in share/\n precise-lite:\n command: |\n .venv/bin/python3 bin/precise.py \"${model}\"\n adapter: |\n wake_adapter_raw.py\n template_args:\n model: \"share/hey_mycroft.tflite\"\n\n # TODO: snowman\n # https://github.com/Thalhammer/snowman/\n\n # ------------------------\n # Voice activity detection\n # ------------------------\n vad:\n\n # https://github.com/snakers4/silero-vad\n # Model included in share/\n silero:\n command: |\n script/speech_prob \"${model}\"\n adapter: |\n vad_adapter_raw.py --rate 16000 --width 2 --channels 1 --samples-per-chunk 512\n template_args:\n model: \"share/silero_vad.onnx\"\n\n # https://pypi.org/project/webrtcvad/\n webrtcvad:\n command: |\n script/speech_prob ${sensitivity}\n adapter: |\n vad_adapter_raw.py --rate 16000 --width 2 --channels 1 --samples-per-chunk 480\n template_args:\n sensitivity: 3\n\n # Uses rms energy threshold.\n # For testing only.\n energy:\n command: |\n bin/energy_speech_prob.py --threshold ${threshold} --width 2 --samples-per-chunk 1024\n adapter: |\n vad_adapter_raw.py --rate 16000 --width 2 --channels 1 --samples-per-chunk 1024\n template_args:\n threshold: 300\n\n # --------------\n # Speech to text\n # --------------\n asr:\n\n # https://alphacephei.com/vosk/\n # Models: https://alphacephei.com/vosk/models\n vosk:\n command: |\n script/raw2text \"${model}\"\n adapter: |\n asr_adapter_raw2text.py --rate 16000 --width 2 --channels 1\n template_args:\n model: \"${data_dir}/vosk-model-small-en-us-0.15\"\n\n # Run server: asr vosk\n vosk.client:\n command: |\n client_unix_socket.py var/run/vosk.socket\n\n # https://stt.readthedocs.io\n # Models: https://coqui.ai/models/\n coqui-stt:\n command: |\n script/raw2text \"${model}\"\n adapter: |\n asr_adapter_raw2text.py --rate 16000 --width 2 --channels 1\n template_args:\n model: \"${data_dir}/english_v1.0.0-large-vocab\"\n\n # Run server: asr coqui-stt\n coqui-stt.client:\n command: |\n client_unix_socket.py var/run/coqui-stt.socket\n\n # https://github.com/cmusphinx/pocketsphinx\n # Models: https://github.com/synesthesiam/voice2json-profiles\n pocketsphinx:\n command: |\n script/raw2text \"${model}\"\n adapter: |\n asr_adapter_raw2text.py --rate 16000 --width 2 --channels 1\n template_args:\n model: \"${data_dir}/en-us_pocketsphinx-cmu\"\n\n # Run server: asr pocketsphinx\n pocketsphinx.client:\n command: |\n client_unix_socket.py var/run/pocketsphinx.socket\n\n # https://github.com/openai/whisper\n # Models: tiny.en,tiny,base.en,base,small.en,small,medium.en,medium,large-v1,large-v2,large\n # Languages: af,am,ar,as,az,ba,be,bg,bn,bo,br,bs,ca,cs,cy,da,de,el,en,es,et,\n # eu,fa,fi,fo,fr,gl,gu,ha,haw,he,hi,hr,ht,hu,hy,id,is,it,ja,jw,ka,kk,km,kn,\n # ko,la,lb,ln,lo,lt,lv,mg,mi,mk,ml,mn,mr,ms,mt,my,ne,nl,nn,no,oc,pa,pl,ps,\n # pt,ro,ru,sa,sd,si,sk,sl,sn,so,sq,sr,su,sv,sw,ta,te,tg,th,tk,tl,tr,tt,uk,\n # ur,uz,vi,yi,yo,zh\n whisper:\n command: |\n script/wav2text --language ${language} --model-directory \"${data_dir}\" \"${model}\" \"{wav_file}\"\n adapter: |\n asr_adapter_wav2text.py\n template_args:\n language: \"en\"\n model: \"tiny.en\"\n\n # Run server: asr whisper\n whisper.client:\n command: |\n client_unix_socket.py var/run/whisper.socket\n\n # https://github.com/ggerganov/whisper.cpp/\n # Models: https://huggingface.co/datasets/ggerganov/whisper.cpp\n whisper-cpp:\n command: |\n script/wav2text \"${model}\" \"{wav_file}\"\n adapter: |\n asr_adapter_wav2text.py\n template_args:\n model: \"${data_dir}/ggml-tiny.en.bin\"\n\n # Run server: asr whisper-cpp\n whisper-cpp.client:\n command: |\n client_unix_socket.py var/run/whisper-cpp.socket\n\n # https://github.com/guillaumekln/faster-whisper/\n # Models: https://github.com/rhasspy/models/releases/tag/v1.0\n # (asr_faster-whisper-*)\n faster-whisper:\n command: |\n script/wav2text --language ${language} \"${model}\" \"{wav_file}\"\n adapter: |\n asr_adapter_wav2text.py\n template_args:\n model: \"${data_dir}/tiny-int8\"\n language: \"en\"\n\n # Run server: asr faster-whisper\n faster-whisper.client:\n command: |\n client_unix_socket.py var/run/faster-whisper.socket\n\n\n # --------------\n # Text to speech\n # --------------\n tts:\n\n # https://github.com/rhasspy/piper/\n # Models: https://github.com/piper/piper/releases/tag/v0.0.2\n piper:\n command: |\n bin/piper --model \"${model}\" --output_file -\n adapter: |\n tts_adapter_text2wav.py\n template_args:\n model: \"${data_dir}/en-us-blizzard_lessac-medium.onnx\"\n install:\n command: |\n script/setup.py --destination '${program_dir}/bin'\n check_file: \"${program_dir}/bin/piper\"\n download:\n command: |\n script/download.py --destination '${data_dir}' '${model}'\n downloads:\n en-us_lessac:\n description: \"U.S. English voice\"\n check_file: \"${data_dir}/en-us-blizzard_lessac-medium.onnx\"\n\n\n # Run server: tts piper\n piper.client:\n command: |\n client_unix_socket.py var/run/piper.socket\n\n # https://github.com/rhasspy/larynx/\n # Models: https://rhasspy.github.io/larynx/\n larynx:\n command: |\n .venv/bin/larynx --voices-dir \"${data_dir}\" --voice \"${voice}\"\n adapter: |\n tts_adapter_text2wav.py\n template_args:\n voice: \"en-us\"\n\n # Run server: tts larynx\n larynx.client:\n command: |\n bin/larynx_client.py ${url} ${voice}\n template_args:\n url: \"http://localhost:5002/process\"\n voice: \"en-us\"\n adapter: |\n tts_adapter_text2wav.py\n\n # https://github.com/espeak-ng/espeak-ng/\n # apt-get install espeak-ng\n espeak-ng:\n command: |\n espeak-ng -v \"${voice}\" --stdin -w \"{temp_file}\"\n adapter: |\n tts_adapter_text2wav.py --temp_file\n template_args:\n voice: \"en-us\"\n\n # http://www.festvox.org/flite/\n # Models: https://github.com/rhasspy/models/releases/tag/v1.0\n # (tts_flite-*)\n flite:\n command: |\n flite -voice \"${voice}\" -o \"{temp_file}\"\n template_args:\n voice: \"cmu_us_slt\"\n adapter: |\n tts_adapter_text2wav.py --temp_file\n\n # http://www.cstr.ed.ac.uk/projects/festival/\n # apt-get install festival festival-\n festival:\n command: |\n text2wave -o \"{temp_file}\" -eval \"(voice_${voice})\"\n template_args:\n voice: \"cmu_us_slt_arctic_hts\"\n adapter: |\n tts_adapter_text2wav.py --temp_file\n\n # https://tts.readthedocs.io\n # Models: see script/list_models\n coqui-tts:\n command: |\n .venv/bin/tts --model_name \"${model}\" --out_path \"{temp_file}\" --text \"{text}\"\n adapter: |\n tts_adapter_text2wav.py --temp_file --text\n template_args:\n model: \"tts_models/en/ljspeech/vits\"\n speaker_id: \"\"\n\n coqui-tts.client:\n command: |\n tts_adapter_http.py \"${url}\" --param speaker_id \"${speaker_id}\"\n template_args:\n url: \"http://localhost:5002/api/tts\"\n speaker_id: \"\"\n\n # http://mary.dfki.de/\n # Models: https://github.com/synesthesiam/opentts/releases/tag/v2.1\n # (marytts-voices.tar.gz)\n marytts:\n command: |\n bin/marytts.py \"${url}\" \"${voice}\"\n template_args:\n url: \"http://localhost:59125/process\"\n voice: \"cmu-slt-hsmm\"\n adapter: |\n tts_adapter_text2wav.py\n\n # https://github.com/mycroftAI/mimic3\n # Models: https://mycroftai.github.io/mimic3-voices/\n mimic3:\n command: |\n .venv/bin/mimic3 --voices-dir \"${data_dir}\" --voice \"${voice}\" --stdout\n adapter: |\n tts_adapter_text2wav.py\n template_args:\n voice: \"apope\"\n\n # Run server: tts mimic3\n mimic3.client:\n command: |\n client_unix_socket.py var/run/mimic3.socket\n\n # ------------------\n # Intent recognition\n # ------------------\n intent:\n\n # Simple regex matching\n regex:\n command: |\n bin/regex.py -i TurnOn \"turn on (the )?(?P.+)\"\n\n # TODO: fsticuffs\n # https://github.com/rhasspy/rhasspy-nlu\n\n # TODO: hassil\n # https://github.com/home-assistant/hassil\n\n # TODO: rapidfuzz\n # https://github.com/rhasspy/rhasspy-fuzzywuzzy\n\n # TODO: snips-nlu\n # https://snips-nlu.readthedocs.io\n\n # ---------------\n # Intent Handling\n # ---------------\n handle:\n\n # Text only: repeats transcript back\n repeat:\n command: |\n cat\n shell: true\n adapter: |\n handle_adapter_text.py\n\n # Text only: send to HA Assist\n # https://www.home-assistant.io/docs/assist\n #\n # 1. Change server url\n # 2. Put long-lived access token in etc/token\n home_assistant:\n command: |\n bin/converse.py --language \"${language}\" \"${url}\" \"${token_file}\"\n adapter: |\n handle_adapter_text.py\n template_args:\n url: \"http://localhost:8123/api/conversation/process\"\n token_file: \"${data_dir}/token\"\n language: \"\"\n\n # Intent only: answer English date/time requests\n date_time:\n command: |\n bin/date_time.py\n adapter: |\n handle_adapter_text.py\n\n # Intent only: produces canned response to regex intent system\n test:\n command: |\n name=\"$(jq -r .slots.name)\"\n echo \"Turned on ${name}.\"\n shell: true\n adapter: |\n handle_adapter_json.py\n\n # ------------\n # Audio output\n # ------------\n snd:\n\n # apt-get install alsa-utils\n aplay:\n command: |\n aplay -q -D \"${device}\" -r 22050 -f S16_LE -c 1 -t raw\n adapter: |\n snd_adapter_raw.py --rate 22050 --width 2 --channels 1\n template_args:\n device: \"default\"\n\n udp_raw:\n command: |\n bin/udp_raw.py --host \"${host}\" --port ${port}\n template_args:\n host: \"127.0.0.1\"\n port: 5001\n\n # -------------------\n # Remote base station\n # -------------------\n remote:\n\n # Sample tool to communicate with websocket API.\n # Use rhasspy3/bin/satellite_run.py\n websocket:\n command: |\n script/run \"${uri}\"\n template_args:\n uri: \"ws://localhost:13331/pipeline/asr-tts\"\n\n# -----------------------------------------------------------------------------\n\nservers:\n asr:\n vosk:\n command: |\n script/server \"${model}\"\n template_args:\n model: \"${data_dir}/vosk-model-small-en-us-0.15\"\n\n coqui-stt:\n command: |\n script/server \"${model}\"\n template_args:\n model: \"${data_dir}/english_v1.0.0-large-vocab\"\n\n pocketsphinx:\n command: |\n script/server \"${model}\"\n template_args:\n model: \"${data_dir}/en-us_pocketsphinx-cmu\"\n\n whisper:\n command: |\n script/server --model-directory \"${data_dir}\" --language ${language} --device ${device} ${model}\n template_args:\n language: \"en\"\n model: \"tiny.en\"\n device: \"cpu\" # or cuda\n\n whisper-cpp:\n command: |\n script/server \"${model}\"\n template_args:\n model: \"${data_dir}/ggml-tiny.en.bin\"\n\n faster-whisper:\n command: |\n script/server --language ${language} --device ${device} \"${model}\"\n template_args:\n language: \"en\"\n model: \"${data_dir}/tiny-int8\"\n device: \"cpu\" # or cuda\n\n tts:\n mimic3:\n command: |\n script/server --voice \"${voice}\" \"${data_dir}\"\n template_args:\n voice: \"en_US/ljspeech_low\"\n\n piper:\n command: |\n script/server \"${model}\"\n template_args:\n model: \"${data_dir}/en-us-blizzard_lessac-medium.onnx\"\n\n larynx:\n command: |\n script/server --voices-dir \"${data_dir}\" --host \"${host}\"\n template_args:\n host: \"127.0.0.1\"\n\n coqui-tts:\n command: |\n script/server\n\n\n# -----------------------------------------------------------------------------\n\n# Example satellites\n# satellites:\n# default:\n# mic:\n# name: arecord\n# wake:\n# name: porcupine1\n# remote:\n# name: websocket\n# snd:\n# name: aplay\n\n# -----------------------------------------------------------------------------\n\n# Example pipelines\npipelines:\n\n # English (default)\n default:\n mic:\n name: arecord\n wake:\n name: porcupine1\n vad:\n name: silero\n asr:\n name: faster-whisper\n # intent:\n # name: regex\n handle:\n name: repeat\n tts:\n name: piper\n snd:\n name: aplay\n\n # # German\n # de:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: de\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/de-thorsten-low.onnx\"\n\n # # French\n # fr:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: fr\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/fr-siwis-low.onnx\"\n\n # # Spanish\n # es:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: es\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/es-carlfm-low.onnx\"\n\n # # Italian\n # it:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: it\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/it-riccardo_fasol-low.onnx\"\n\n # # Catalan\n # ca:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: ca\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/ca-upc_ona-low.onnx\"\n\n # # Danish\n # da:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: da\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/da-nst_talesyntese-medium.onnx\"\n\n # # Dutch\n # nl:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: nl\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/nl-nathalie-low.onnx\"\n\n # # Norwegian\n # no:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: no\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/no-talesyntese-medium.onnx\"\n\n # # Ukrainian\n # uk:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: uk\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/uk-lada-low.onnx\"\n\n # # Vietnamese\n # vi:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: vi\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/vi-vivos-low.onnx\"\n\n # # Chinese\n # zh:\n # inherit: default\n # asr:\n # name: faster-whisper\n # template_args:\n # language: zh\n # tts:\n # name: piper\n # template_args:\n # model: \"${data_dir}/zh-cn-huayan-low.onnx\"\n" }, { "path": "rhasspy3/core.py", "content": "import logging\nfrom dataclasses import dataclass\nfrom pathlib import Path\nfrom typing import Any, Dict, Union\n\nfrom .config import Config\nfrom .util import merge_dict\nfrom .util.jaml import safe_load\n\n_DIR = Path(__file__).parent\n_DEFAULT_CONFIG = _DIR / \"configuration.yaml\"\n_LOGGER = logging.getLogger(__name__)\n\n\n@dataclass\nclass Rhasspy:\n config: Config\n config_dir: Path\n base_dir: Path\n config_dict: Dict[str, Any]\n\n @property\n def programs_dir(self) -> Path:\n \"\"\"Directory where programs are installed.\"\"\"\n return self.config_dir / \"programs\"\n\n @property\n def data_dir(self) -> Path:\n \"\"\"Directory where models are downloaded.\"\"\"\n return self.config_dir / \"data\"\n\n @staticmethod\n def load(config_dir: Union[str, Path]) -> \"Rhasspy\":\n \"\"\"Load and merge configuration.yaml files from rhasspy3 and config dir.\"\"\"\n config_dir = Path(config_dir)\n config_paths = [\n _DEFAULT_CONFIG,\n config_dir / \"configuration.yaml\",\n ]\n config_dict: Dict[str, Any] = {}\n\n for config_path in config_paths:\n if config_path.exists():\n _LOGGER.debug(\"Loading config from %s\", config_path)\n with config_path.open(encoding=\"utf-8\") as config_file:\n merge_dict(config_dict, safe_load(config_file))\n else:\n _LOGGER.debug(\"Skipping %s\", config_path)\n\n return Rhasspy(\n config=Config.from_dict(config_dict),\n config_dir=config_dir,\n config_dict=config_dict,\n base_dir=_DIR.parent,\n )\n" }, { "path": "rhasspy3/event.py", "content": "import asyncio\nimport json\nimport sys\nfrom abc import ABC, abstractmethod\nfrom dataclasses import dataclass, field\nfrom typing import IO, Any, Dict, Iterable, Optional\n\n_TYPE = \"type\"\n_DATA = \"data\"\n_PAYLOAD_LENGTH = \"payload_length\"\n_NEWLINE = \"\\n\".encode()\n\n\n@dataclass\nclass Event:\n type: str\n data: Dict[str, Any] = field(default_factory=dict)\n payload: Optional[bytes] = None\n\n def to_dict(self) -> Dict[str, Any]:\n return {_TYPE: self.type, _DATA: self.data}\n\n @staticmethod\n def from_dict(event_dict: Dict[str, Any]) -> \"Event\":\n return Event(type=event_dict[\"type\"], data=event_dict.get(\"data\", {}))\n\n\nclass Eventable(ABC):\n @abstractmethod\n def event(self) -> Event:\n pass\n\n @staticmethod\n @abstractmethod\n def is_type(event_type: str) -> bool:\n pass\n\n def to_dict(self) -> Dict[str, Any]:\n return self.event().data\n\n\nasync def async_read_event(reader: asyncio.StreamReader) -> Optional[Event]:\n try:\n json_line = await reader.readline()\n if not json_line:\n return None\n\n event_dict = json.loads(json_line)\n payload_length = event_dict.get(_PAYLOAD_LENGTH)\n\n payload: Optional[bytes] = None\n if payload_length is not None:\n payload = await reader.readexactly(payload_length)\n\n return Event(\n type=event_dict[_TYPE], data=event_dict.get(_DATA), payload=payload\n )\n except KeyboardInterrupt:\n pass\n\n return None\n\n\nasync def async_write_event(event: Event, writer: asyncio.StreamWriter):\n event_dict: Dict[str, Any] = event.to_dict()\n if event.payload:\n event_dict[_PAYLOAD_LENGTH] = len(event.payload)\n\n json_line = json.dumps(event_dict, ensure_ascii=False)\n\n try:\n writer.writelines((json_line.encode(), _NEWLINE))\n\n if event.payload:\n writer.write(event.payload)\n\n await writer.drain()\n except KeyboardInterrupt:\n pass\n\n\nasync def async_write_events(events: Iterable[Event], writer: asyncio.StreamWriter):\n coros = []\n for event in events:\n event_dict: Dict[str, Any] = event.to_dict()\n if event.payload:\n event_dict[_PAYLOAD_LENGTH] = len(event.payload)\n\n json_line = json.dumps(event_dict, ensure_ascii=False)\n writer.writelines((json_line.encode(), _NEWLINE))\n\n if event.payload:\n writer.write(event.payload)\n\n coros.append(writer.drain())\n\n try:\n await asyncio.gather(*coros)\n except KeyboardInterrupt:\n pass\n\n\ndef read_event(reader: Optional[IO[bytes]] = None) -> Optional[Event]:\n if reader is None:\n reader = sys.stdin.buffer\n\n try:\n json_line = reader.readline()\n\n if not json_line:\n return None\n\n event_dict = json.loads(json_line)\n payload_length = event_dict.get(_PAYLOAD_LENGTH)\n\n payload: Optional[bytes] = None\n if payload_length is not None:\n payload = reader.read(payload_length)\n\n return Event(\n type=event_dict[_TYPE], data=event_dict.get(_DATA), payload=payload\n )\n except KeyboardInterrupt:\n pass\n\n return None\n\n\ndef write_event(event: Event, writer: Optional[IO[bytes]] = None):\n if writer is None:\n writer = sys.stdout.buffer\n\n event_dict: Dict[str, Any] = event.to_dict()\n if event.payload:\n event_dict[_PAYLOAD_LENGTH] = len(event.payload)\n\n json_line = json.dumps(event_dict, ensure_ascii=False)\n\n try:\n writer.writelines((json_line.encode(), _NEWLINE))\n\n if event.payload:\n writer.write(event.payload)\n\n writer.flush()\n except KeyboardInterrupt:\n pass\n" }, { "path": "rhasspy3/handle.py", "content": "\"\"\"Intent recognition and handling.\"\"\"\nimport logging\nfrom dataclasses import dataclass\nfrom typing import Any, Dict, Optional, Union\n\nfrom .asr import Transcript\nfrom .config import PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event, async_write_event\nfrom .intent import Intent, NotRecognized\nfrom .program import create_process\n\nDOMAIN = \"handle\"\n_HANDLED_TYPE = \"handled\"\n_NOT_HANDLED_TYPE = \"not-handled\"\n\n_LOGGER = logging.getLogger(__name__)\n\n\n@dataclass\nclass Handled(Eventable):\n text: Optional[str] = None\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _HANDLED_TYPE\n\n def event(self) -> Event:\n data: Dict[str, Any] = {}\n if self.text is not None:\n data[\"text\"] = self.text\n\n return Event(type=_HANDLED_TYPE, data=data)\n\n @staticmethod\n def from_event(event: Event) -> \"Handled\":\n assert event.data is not None\n return Handled(text=event.data.get(\"text\"))\n\n\n@dataclass\nclass NotHandled(Eventable):\n text: Optional[str] = None\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _NOT_HANDLED_TYPE\n\n def event(self) -> Event:\n data: Dict[str, Any] = {}\n if self.text is not None:\n data[\"text\"] = self.text\n\n return Event(type=_NOT_HANDLED_TYPE, data=data)\n\n @staticmethod\n def from_event(event: Event) -> \"NotHandled\":\n assert event.data is not None\n return NotHandled(text=event.data.get(\"text\"))\n\n\nasync def handle(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n handle_input: Union[Intent, NotRecognized, Transcript],\n) -> Optional[Union[Handled, NotHandled]]:\n handle_result: Optional[Union[Handled, NotHandled]] = None\n async with (await create_process(rhasspy, DOMAIN, program)) as handle_proc:\n assert handle_proc.stdin is not None\n assert handle_proc.stdout is not None\n\n _LOGGER.debug(\"handle: input=%s\", handle_input)\n await async_write_event(handle_input.event(), handle_proc.stdin)\n while True:\n event = await async_read_event(handle_proc.stdout)\n if event is None:\n break\n\n if Handled.is_type(event.type):\n handle_result = Handled.from_event(event)\n elif NotHandled.is_type(event.type):\n handle_result = NotHandled.from_event(event)\n\n _LOGGER.debug(\"handle: %s\", handle_result)\n\n return handle_result\n" }, { "path": "rhasspy3/intent.py", "content": "\"\"\"Intent recognition and handling.\"\"\"\nimport logging\nfrom dataclasses import asdict, dataclass, field\nfrom typing import Any, Dict, List, Optional, Union\n\nfrom .config import PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event, async_write_event\nfrom .program import create_process\n\nDOMAIN = \"intent\"\n_RECOGNIZE_TYPE = \"recognize\"\n_INTENT_TYPE = \"intent\"\n_NOT_RECOGNIZED_TYPE = \"not-recognized\"\n\n_LOGGER = logging.getLogger(__name__)\n\n\n@dataclass\nclass Entity:\n name: str\n value: Optional[Any] = None\n\n\n@dataclass\nclass Recognize(Eventable):\n text: str\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _RECOGNIZE_TYPE\n\n def event(self) -> Event:\n data: Dict[str, Any] = {\"text\": self.text}\n return Event(type=_RECOGNIZE_TYPE, data=data)\n\n @staticmethod\n def from_event(event: Event) -> \"Recognize\":\n assert event.data is not None\n return Recognize(text=event.data[\"text\"])\n\n\n@dataclass\nclass Intent(Eventable):\n name: str\n entities: List[Entity] = field(default_factory=list)\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _INTENT_TYPE\n\n def event(self) -> Event:\n data: Dict[str, Any] = {\"name\": self.name}\n if self.entities:\n data[\"entities\"] = [asdict(entity) for entity in self.entities]\n\n return Event(type=_INTENT_TYPE, data=data)\n\n @staticmethod\n def from_dict(data: Dict[str, Any]) -> \"Intent\":\n entity_dicts = data.get(\"entities\")\n if entity_dicts:\n entities: List[Entity] = [\n Entity(**entity_dict) for entity_dict in entity_dicts\n ]\n else:\n entities = []\n\n return Intent(name=data[\"name\"], entities=entities)\n\n @staticmethod\n def from_event(event: Event) -> \"Intent\":\n assert event.data is not None\n return Intent.from_dict(event.data)\n\n def to_rhasspy(self) -> Dict[str, Any]:\n return {\n \"intent\": {\n \"name\": self.name,\n },\n \"entities\": [\n {\"entity\": entity.name, \"value\": entity.value}\n for entity in self.entities\n ],\n \"slots\": {entity.name: entity.value for entity in self.entities},\n }\n\n\n@dataclass\nclass NotRecognized(Eventable):\n text: Optional[str] = None\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _NOT_RECOGNIZED_TYPE\n\n def event(self) -> Event:\n data: Dict[str, Any] = {}\n if self.text is not None:\n data[\"text\"] = self.text\n\n return Event(type=_NOT_RECOGNIZED_TYPE, data=data)\n\n @staticmethod\n def from_event(event: Event) -> \"NotRecognized\":\n assert event.data is not None\n return NotRecognized(text=event.data.get(\"text\"))\n\n\nasync def recognize(\n rhasspy: Rhasspy, program: Union[str, PipelineProgramConfig], text: str\n) -> Optional[Union[Intent, NotRecognized]]:\n result: Optional[Union[Intent, NotRecognized]] = None\n async with (await create_process(rhasspy, DOMAIN, program)) as intent_proc:\n assert intent_proc.stdin is not None\n assert intent_proc.stdout is not None\n\n _LOGGER.debug(\"recognize: text='%s'\", text)\n await async_write_event(Recognize(text=text).event(), intent_proc.stdin)\n while True:\n intent_event = await async_read_event(intent_proc.stdout)\n if intent_event is None:\n break\n\n if Intent.is_type(intent_event.type):\n result = Intent.from_event(intent_event)\n break\n\n if NotRecognized.is_type(intent_event.type):\n result = NotRecognized.from_event(intent_event)\n break\n\n _LOGGER.debug(\"recognize: %s\", result)\n\n return result\n" }, { "path": "rhasspy3/mic.py", "content": "\"\"\"Audio input from a microphone.\"\"\"\nDOMAIN = \"mic\"\n" }, { "path": "rhasspy3/pipeline.py", "content": "\"\"\"Full voice loop (pipeline).\"\"\"\nimport io\nimport logging\nfrom collections import deque\nfrom dataclasses import dataclass, fields\nfrom enum import Enum\nfrom typing import IO, Any, Deque, Dict, Optional, Union\n\nfrom .asr import DOMAIN as ASR_DOMAIN\nfrom .asr import Transcript, transcribe\nfrom .config import CommandConfig, PipelineConfig, PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event\nfrom .handle import Handled, NotHandled, handle\nfrom .intent import Intent, NotRecognized, recognize\nfrom .mic import DOMAIN as MIC_DOMAIN\nfrom .program import create_process, run_command\nfrom .snd import play\nfrom .tts import synthesize\nfrom .util.dataclasses_json import DataClassJsonMixin\nfrom .vad import segment\nfrom .wake import Detection, detect\n\n_LOGGER = logging.getLogger(__name__)\n\n\n@dataclass\nclass PipelineResult(DataClassJsonMixin):\n \"\"\"Result of running all or part of a pipeline.\"\"\"\n\n wake_detection: Optional[Detection] = None\n asr_transcript: Optional[Transcript] = None\n intent_result: Optional[Union[Intent, NotRecognized]] = None\n handle_result: Optional[Union[Handled, NotHandled]] = None\n\n def to_event_dict(self) -> Dict[str, Any]:\n event_dict: Dict[str, Any] = {}\n for field in fields(self):\n value = getattr(self, field.name)\n if value is None:\n event_dict[field.name] = {}\n else:\n assert isinstance(value, Eventable)\n event_dict[field.name] = value.event().to_dict()\n\n return event_dict\n\n\nclass StopAfterDomain(str, Enum):\n WAKE = \"wake\"\n ASR = \"asr\"\n INTENT = \"intent\"\n HANDLE = \"handle\"\n TTS = \"tts\"\n\n\nasync def run(\n rhasspy: Rhasspy,\n pipeline: Union[str, PipelineConfig],\n samples_per_chunk: int,\n asr_chunks_to_buffer: int = 0,\n mic_program: Optional[Union[str, PipelineProgramConfig]] = None,\n wake_program: Optional[Union[str, PipelineProgramConfig]] = None,\n wake_detection: Optional[Detection] = None,\n asr_program: Optional[Union[str, PipelineProgramConfig]] = None,\n asr_wav_in: Optional[IO[bytes]] = None,\n asr_transcript: Optional[Transcript] = None,\n vad_program: Optional[Union[str, PipelineProgramConfig]] = None,\n intent_result: Optional[Union[Intent, NotRecognized]] = None,\n intent_program: Optional[Union[str, PipelineProgramConfig]] = None,\n handle_result: Optional[Union[Handled, NotHandled]] = None,\n handle_program: Optional[Union[str, PipelineProgramConfig]] = None,\n tts_wav_in: Optional[IO[bytes]] = None,\n tts_program: Optional[Union[str, PipelineProgramConfig]] = None,\n snd_program: Optional[Union[str, PipelineProgramConfig]] = None,\n stop_after: Optional[StopAfterDomain] = None,\n) -> PipelineResult:\n \"\"\"Run a full or partial pipeline.\"\"\"\n pipeline_result = PipelineResult()\n\n if isinstance(pipeline, str):\n pipeline = rhasspy.config.pipelines[pipeline]\n\n mic_program = mic_program or pipeline.mic\n\n wake_program = wake_program or pipeline.wake\n wake_after = pipeline.wake.after if pipeline.wake else None\n\n asr_program = asr_program or pipeline.asr\n asr_after = pipeline.asr.after if pipeline.asr else None\n\n vad_program = vad_program or pipeline.vad\n intent_program = intent_program or pipeline.intent\n handle_program = handle_program or pipeline.handle\n tts_program = tts_program or pipeline.tts\n snd_program = snd_program or pipeline.snd\n\n skip_asr = (\n (intent_result is not None)\n or (handle_result is not None)\n or (tts_wav_in is not None)\n )\n\n if not skip_asr:\n # Speech to text\n if asr_wav_in is not None:\n # WAV input\n if stop_after == StopAfterDomain.WAKE:\n return pipeline_result\n\n asr_wav_in.seek(0)\n assert asr_program is not None, \"No asr program\"\n asr_transcript = await transcribe(\n rhasspy, asr_program, asr_wav_in, samples_per_chunk\n )\n\n if asr_after is not None:\n await run_command(rhasspy, asr_after)\n elif asr_transcript is None:\n # Mic input\n assert mic_program is not None, \"No asr program\"\n\n if wake_program is None:\n # No wake\n assert asr_program is not None, \"No asr program\"\n assert vad_program is not None, \"No vad program\"\n await _mic_asr(\n rhasspy, mic_program, asr_program, vad_program, pipeline_result\n )\n elif stop_after == StopAfterDomain.WAKE:\n # Audio input, wake word detection, segmentation, speech to text\n assert wake_program is not None, \"No vad program\"\n await _mic_wake(\n rhasspy,\n mic_program,\n wake_program,\n pipeline_result,\n wake_detection=wake_detection,\n )\n return pipeline_result\n else:\n assert wake_program is not None, \"No vad program\"\n assert asr_program is not None, \"No asr program\"\n assert vad_program is not None, \"No vad program\"\n await _mic_wake_asr(\n rhasspy,\n mic_program,\n wake_program,\n asr_program,\n vad_program,\n pipeline_result,\n asr_chunks_to_buffer=asr_chunks_to_buffer,\n wake_detection=wake_detection,\n wake_after=wake_after,\n )\n\n if asr_after is not None:\n await run_command(rhasspy, asr_after)\n\n asr_transcript = pipeline_result.asr_transcript\n pipeline_result.asr_transcript = asr_transcript\n\n if (stop_after == StopAfterDomain.ASR) or (\n (intent_program is None) and (handle_program is None)\n ):\n return pipeline_result\n\n # Text to intent\n if (asr_transcript is not None) and (intent_program is not None):\n pipeline_result.asr_transcript = asr_transcript\n intent_result = await recognize(\n rhasspy, intent_program, asr_transcript.text or \"\"\n )\n pipeline_result.intent_result = intent_result\n\n # Handle intent\n handle_input: Optional[Union[Intent, NotRecognized, Transcript]] = None\n if intent_result is not None:\n pipeline_result.intent_result = intent_result\n handle_input = intent_result\n elif asr_transcript is not None:\n handle_input = asr_transcript\n\n if (stop_after == StopAfterDomain.INTENT) or (handle_program is None):\n return pipeline_result\n\n if (handle_input is not None) and (handle_result is None):\n assert handle_program is not None, \"Pipeline is missing handle\"\n handle_result = await handle(rhasspy, handle_program, handle_input)\n pipeline_result.handle_result = handle_result\n\n if (stop_after == StopAfterDomain.HANDLE) or (tts_program is None):\n return pipeline_result\n\n # Text to speech\n if handle_result is not None:\n pipeline_result.handle_result = handle_result\n if handle_result.text:\n assert tts_program is not None, \"Pipeline is missing tts\"\n tts_wav_in = io.BytesIO()\n await synthesize(rhasspy, tts_program, handle_result.text, tts_wav_in)\n else:\n _LOGGER.debug(\"No text returned from handle\")\n\n if (stop_after == StopAfterDomain.TTS) or (snd_program is None):\n return pipeline_result\n\n # Audio output\n if tts_wav_in is not None:\n tts_wav_in.seek(0)\n assert snd_program is not None, \"Pipeline is missing snd\"\n await play(rhasspy, snd_program, tts_wav_in, samples_per_chunk)\n\n return pipeline_result\n\n\nasync def _mic_wake(\n rhasspy: Rhasspy,\n mic_program: Union[str, PipelineProgramConfig],\n wake_program: Union[str, PipelineProgramConfig],\n pipeline_result: PipelineResult,\n wake_detection: Optional[Detection] = None,\n):\n \"\"\"Just wake word detection.\"\"\"\n async with (await create_process(rhasspy, MIC_DOMAIN, mic_program)) as mic_proc:\n assert mic_proc.stdout is not None\n if wake_detection is None:\n wake_detection = await detect(\n rhasspy,\n wake_program,\n mic_proc.stdout,\n )\n\n if wake_detection is not None:\n pipeline_result.wake_detection = wake_detection\n else:\n _LOGGER.debug(\"run: no wake word detected\")\n\n\nasync def _mic_asr(\n rhasspy: Rhasspy,\n mic_program: Union[str, PipelineProgramConfig],\n asr_program: Union[str, PipelineProgramConfig],\n vad_program: Union[str, PipelineProgramConfig],\n pipeline_result: PipelineResult,\n asr_chunks_to_buffer: int = 0,\n):\n \"\"\"Just asr transcription (+ silence detection).\"\"\"\n async with (await create_process(rhasspy, MIC_DOMAIN, mic_program)) as mic_proc, (\n await create_process(rhasspy, ASR_DOMAIN, asr_program)\n ) as asr_proc:\n assert mic_proc.stdout is not None\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n\n await segment(\n rhasspy,\n vad_program,\n mic_proc.stdout,\n asr_proc.stdin,\n )\n while True:\n asr_event = await async_read_event(asr_proc.stdout)\n if asr_event is None:\n break\n\n if Transcript.is_type(asr_event.type):\n pipeline_result.asr_transcript = Transcript.from_event(asr_event)\n break\n\n\nasync def _mic_wake_asr(\n rhasspy: Rhasspy,\n mic_program: Union[str, PipelineProgramConfig],\n wake_program: Union[str, PipelineProgramConfig],\n asr_program: Union[str, PipelineProgramConfig],\n vad_program: Union[str, PipelineProgramConfig],\n pipeline_result: PipelineResult,\n asr_chunks_to_buffer: int = 0,\n wake_detection: Optional[Detection] = None,\n wake_after: Optional[CommandConfig] = None,\n):\n \"\"\"Wake word detect + asr transcription (+ silence detection).\"\"\"\n chunk_buffer: Optional[Deque[Event]] = (\n deque(maxlen=asr_chunks_to_buffer) if asr_chunks_to_buffer > 0 else None\n )\n\n async with (await create_process(rhasspy, MIC_DOMAIN, mic_program)) as mic_proc, (\n await create_process(rhasspy, ASR_DOMAIN, asr_program)\n ) as asr_proc:\n assert mic_proc.stdout is not None\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n\n if wake_detection is None:\n wake_detection = await detect(\n rhasspy, wake_program, mic_proc.stdout, chunk_buffer\n )\n\n if wake_detection is not None:\n if wake_after is not None:\n await run_command(rhasspy, wake_after)\n\n pipeline_result.wake_detection = wake_detection\n await segment(\n rhasspy,\n vad_program,\n mic_proc.stdout,\n asr_proc.stdin,\n chunk_buffer,\n )\n while True:\n asr_event = await async_read_event(asr_proc.stdout)\n if asr_event is None:\n break\n\n if Transcript.is_type(asr_event.type):\n pipeline_result.asr_transcript = Transcript.from_event(asr_event)\n break\n else:\n _LOGGER.debug(\"run: no wake word detected\")\n" }, { "path": "rhasspy3/program.py", "content": "\"\"\"Utilities for creating processes.\"\"\"\nimport asyncio\nimport logging\nimport os\nimport shlex\nimport string\nfrom asyncio.subprocess import PIPE, Process\nfrom typing import Optional, Union\n\nfrom .config import CommandConfig, PipelineProgramConfig, ProgramConfig\nfrom .core import Rhasspy\nfrom .util import merge_dict\n\n_LOGGER = logging.getLogger(__name__)\n\n\nclass MissingProgramConfigError(Exception):\n pass\n\n\nclass ProcessContextManager:\n \"\"\"Wrapper for an async process that terminates on exit.\"\"\"\n\n def __init__(self, proc: Process, name: str):\n self.proc = proc\n self.name = name\n\n async def __aenter__(self):\n return self.proc\n\n async def __aexit__(self, exc_type, exc, tb):\n try:\n if self.proc.returncode is None:\n self.proc.terminate()\n await self.proc.wait()\n except ProcessLookupError:\n # Expected when process has already exited\n pass\n except Exception:\n _LOGGER.exception(\"Unexpected error stopping process: %s\", self.name)\n\n\nasync def create_process(\n rhasspy: Rhasspy, domain: str, name: Union[str, PipelineProgramConfig]\n) -> ProcessContextManager:\n pipeline_config: Optional[PipelineProgramConfig] = None\n if isinstance(name, PipelineProgramConfig):\n pipeline_config = name\n name = pipeline_config.name\n\n assert name, f\"No program name for domain {domain}\"\n\n # The \".\" is special in program names:\n # it means to use the directory of \"base\" in ..\n #\n # This is used for .client programs, which are just scripts in the\n # \"base\" directory that communicate with their respective servers.\n if \".\" in name:\n base_name = name.split(\".\", maxsplit=1)[0]\n else:\n base_name = name\n\n program_config: Optional[ProgramConfig] = rhasspy.config.programs.get(\n domain, {}\n ).get(name)\n assert program_config is not None, f\"No config for program {domain}/{name}\"\n assert isinstance(program_config, ProgramConfig)\n\n # Directory where this program is installed\n program_dir = rhasspy.programs_dir / domain / base_name\n\n # Directory where this program should store data\n data_dir = rhasspy.data_dir / domain / base_name\n\n # ${variables} available within program/pipeline template_args\n default_mapping = {\n \"program_dir\": str(program_dir.absolute()),\n \"data_dir\": str(data_dir.absolute()),\n }\n\n command_str = program_config.command.strip()\n command_mapping = dict(default_mapping)\n if program_config.template_args:\n # Substitute within program template args\n args_mapping = dict(program_config.template_args)\n for arg_name, arg_str in args_mapping.items():\n if not isinstance(arg_str, str):\n continue\n\n arg_template = string.Template(arg_str)\n args_mapping[arg_name] = arg_template.safe_substitute(default_mapping)\n\n command_mapping.update(args_mapping)\n\n if pipeline_config is not None:\n if pipeline_config.template_args:\n # Substitute within pipeline template args\n args_mapping = dict(pipeline_config.template_args)\n for arg_name, arg_str in args_mapping.items():\n if not isinstance(arg_str, str):\n continue\n\n arg_template = string.Template(arg_str)\n args_mapping[arg_name] = arg_template.safe_substitute(default_mapping)\n\n merge_dict(command_mapping, args_mapping)\n\n # Substitute template args\n command_template = string.Template(command_str)\n command_str = command_template.safe_substitute(command_mapping)\n\n working_dir = rhasspy.programs_dir / domain / base_name\n env = dict(os.environ)\n\n # Add rhasspy3/bin to $PATH\n env[\"PATH\"] = f'{rhasspy.base_dir}/bin:${env[\"PATH\"]}'\n\n # Ensure stdout is flushed for Python programs\n env[\"PYTHONUNBUFFERED\"] = \"1\"\n\n cwd = working_dir if working_dir.is_dir() else None\n\n if program_config.shell:\n if program_config.adapter:\n program, *args = shlex.split(program_config.adapter)\n args.append(\"--shell\")\n args.append(command_str)\n\n _LOGGER.debug(\"(shell): %s %s\", program, args)\n proc = await asyncio.create_subprocess_exec(\n program,\n *args,\n stdin=PIPE,\n stdout=PIPE,\n cwd=cwd,\n env=env,\n )\n else:\n _LOGGER.debug(\"(shell): %s %s\", program, args)\n proc = await asyncio.create_subprocess_shell(\n command_str,\n stdin=PIPE,\n stdout=PIPE,\n cwd=cwd,\n env=env,\n )\n else:\n if program_config.adapter:\n program, *args = shlex.split(program_config.adapter)\n args.append(command_str)\n else:\n program, *args = shlex.split(command_str)\n\n _LOGGER.debug(\"%s %s\", program, args)\n proc = await asyncio.create_subprocess_exec(\n program,\n *args,\n stdin=PIPE,\n stdout=PIPE,\n cwd=cwd,\n env=env,\n )\n\n return ProcessContextManager(proc, name=name)\n\n\nasync def run_command(rhasspy: Rhasspy, command_config: CommandConfig) -> int:\n env = dict(os.environ)\n\n # Add rhasspy3/bin to $PATH\n env[\"PATH\"] = f'{rhasspy.base_dir}/bin:${env[\"PATH\"]}'\n\n # Ensure stdout is flushed for Python programs\n env[\"PYTHONUNBUFFERED\"] = \"1\"\n\n if command_config.shell:\n proc = await asyncio.create_subprocess_shell(\n command_config.command,\n env=env,\n )\n else:\n program, *args = shlex.split(command_config.command)\n proc = await asyncio.create_subprocess_exec(\n program,\n *args,\n env=env,\n )\n\n await proc.wait()\n assert proc.returncode is not None\n\n return proc.returncode\n" }, { "path": "rhasspy3/py.typed", "content": "" }, { "path": "rhasspy3/remote.py", "content": "\"\"\"Remote communication with a base station.\"\"\"\nDOMAIN = \"remote\"\n" }, { "path": "rhasspy3/snd.py", "content": "\"\"\"Audio output to speakers.\"\"\"\nimport wave\nfrom dataclasses import dataclass\nfrom typing import IO, AsyncIterable, Optional, Union\n\nfrom .audio import AudioChunk, AudioStop, wav_to_chunks\nfrom .config import PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event, async_write_event\nfrom .program import create_process\n\nDOMAIN = \"snd\"\n_PLAYED_TYPE = \"played\"\n\n\n@dataclass\nclass Played(Eventable):\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _PLAYED_TYPE\n\n def event(self) -> Event:\n return Event(type=_PLAYED_TYPE)\n\n @staticmethod\n def from_event(event: Event) -> \"Played\":\n return Played()\n\n\nasync def play(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n wav_in: IO[bytes],\n samples_per_chunk: int,\n) -> Optional[Played]:\n wav_file: wave.Wave_read = wave.open(wav_in, \"rb\")\n with wav_file:\n async with (await create_process(rhasspy, DOMAIN, program)) as snd_proc:\n assert snd_proc.stdin is not None\n assert snd_proc.stdout is not None\n\n timestamp: Optional[int] = None\n for chunk in wav_to_chunks(wav_file, samples_per_chunk=samples_per_chunk):\n await async_write_event(chunk.event(), snd_proc.stdin)\n timestamp = chunk.timestamp\n\n await async_write_event(\n AudioStop(timestamp=timestamp).event(), snd_proc.stdin\n )\n\n # Wait for confimation\n while True:\n event = await async_read_event(snd_proc.stdout)\n if event is None:\n break\n\n if Played.is_type(event.type):\n return Played.from_event(event)\n\n return None\n\n\nasync def play_stream(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n audio_stream: AsyncIterable[bytes],\n rate: int,\n width: int,\n channels: int,\n) -> Optional[Played]:\n async with (await create_process(rhasspy, DOMAIN, program)) as snd_proc:\n assert snd_proc.stdin is not None\n assert snd_proc.stdout is not None\n\n async for audio_bytes in audio_stream:\n chunk = AudioChunk(rate, width, channels, audio_bytes)\n await async_write_event(chunk.event(), snd_proc.stdin)\n\n await async_write_event(AudioStop().event(), snd_proc.stdin)\n\n # Wait for confimation\n while True:\n event = await async_read_event(snd_proc.stdout)\n if event is None:\n break\n\n if Played.is_type(event.type):\n return Played.from_event(event)\n\n return None\n" }, { "path": "rhasspy3/tts.py", "content": "\"\"\"Text to speech.\"\"\"\nimport wave\nfrom dataclasses import dataclass\nfrom typing import IO, AsyncIterable, Union\n\nfrom .audio import AudioChunk, AudioStart, AudioStop\nfrom .config import PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event, async_write_event\nfrom .program import create_process\n\nDOMAIN = \"tts\"\n_SYNTHESIZE_TYPE = \"synthesize\"\n\n\n@dataclass\nclass Synthesize(Eventable):\n \"\"\"Request to synthesize audio from text.\"\"\"\n\n text: str\n \"\"\"Text to synthesize.\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _SYNTHESIZE_TYPE\n\n def event(self) -> Event:\n return Event(type=_SYNTHESIZE_TYPE, data={\"text\": self.text})\n\n @staticmethod\n def from_event(event: Event) -> \"Synthesize\":\n assert event.data is not None\n return Synthesize(text=event.data[\"text\"])\n\n\nasync def synthesize(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n text: str,\n wav_out: IO[bytes],\n):\n \"\"\"Synthesize audio from text to WAV output.\"\"\"\n async with (await create_process(rhasspy, DOMAIN, program)) as tts_proc:\n assert tts_proc.stdin is not None\n assert tts_proc.stdout is not None\n\n await async_write_event(Synthesize(text=text).event(), tts_proc.stdin)\n\n wav_file: wave.Wave_write = wave.open(wav_out, \"wb\")\n wav_params_set = False\n with wav_file:\n while True:\n event = await async_read_event(tts_proc.stdout)\n if event is None:\n break\n\n if AudioStart.is_type(event.type):\n if not wav_params_set:\n start = AudioStart.from_event(event)\n wav_file.setframerate(start.rate)\n wav_file.setsampwidth(start.width)\n wav_file.setnchannels(start.channels)\n wav_params_set = True\n elif AudioChunk.is_type(event.type):\n chunk = AudioChunk.from_event(event)\n\n if not wav_params_set:\n wav_file.setframerate(chunk.rate)\n wav_file.setsampwidth(chunk.width)\n wav_file.setnchannels(chunk.channels)\n wav_params_set = True\n\n wav_file.writeframes(chunk.audio)\n elif AudioStop.is_type(event.type):\n break\n\n\nasync def synthesize_stream(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n text: str,\n) -> AsyncIterable[AudioChunk]:\n \"\"\"Synthesize audio from text to a raw stream.\"\"\"\n async with (await create_process(rhasspy, DOMAIN, program)) as tts_proc:\n assert tts_proc.stdin is not None\n assert tts_proc.stdout is not None\n\n await async_write_event(Synthesize(text=text).event(), tts_proc.stdin)\n\n while True:\n event = await async_read_event(tts_proc.stdout)\n if event is None:\n break\n\n if AudioChunk.is_type(event.type):\n yield AudioChunk.from_event(event)\n elif AudioStop.is_type(event.type):\n break\n" }, { "path": "rhasspy3/util/__init__.py", "content": "import collections\n\n\ndef merge_dict(base_dict, new_dict):\n \"\"\"Merges new_dict into base_dict.\"\"\"\n for key, value in new_dict.items():\n if key in base_dict:\n old_value = base_dict[key]\n if isinstance(old_value, collections.abc.MutableMapping):\n # Combine dictionary\n assert isinstance(\n value, collections.abc.Mapping\n ), f\"Not a dict: {value}\"\n merge_dict(old_value, value)\n elif isinstance(old_value, collections.abc.MutableSequence):\n # Combine list\n assert isinstance(\n value, collections.abc.Sequence\n ), f\"Not a list: {value}\"\n old_value.extend(value)\n else:\n # Overwrite\n base_dict[key] = value\n else:\n base_dict[key] = value\n" }, { "path": "rhasspy3/util/dataclasses_json.py", "content": "\"\"\"Implement a tiny subset of dataclasses_json for config.\"\"\"\nfrom collections.abc import Mapping, Sequence\nfrom dataclasses import asdict, fields, is_dataclass\nfrom typing import Any, Dict, Type\n\n\nclass DataClassJsonMixin:\n \"\"\"Adds from_dict to dataclass.\"\"\"\n\n @classmethod\n def from_dict(cls, data: Dict[str, Any]) -> Any:\n \"\"\"Parse dataclasses recursively.\"\"\"\n kwargs: Dict[str, Any] = {}\n\n cls_fields = {field.name: field for field in fields(cls)}\n for key, value in data.items():\n field = cls_fields[key]\n if is_dataclass(field.type):\n assert issubclass(field.type, DataClassJsonMixin), field.type\n kwargs[key] = field.type.from_dict(value)\n else:\n kwargs[key] = _decode(value, field.type)\n\n return cls(**kwargs)\n\n def to_dict(self) -> Dict[str, Any]:\n \"\"\"Alias for asdict.\"\"\"\n return asdict(self)\n\n\ndef _decode(value: Any, target_type: Type) -> Any:\n \"\"\"Decode value using (possibly generic) type.\"\"\"\n if is_dataclass(target_type):\n assert issubclass(target_type, DataClassJsonMixin), target_type\n return target_type.from_dict(value) if value is not None else None\n\n if hasattr(target_type, \"__args__\"):\n # Optional[T]\n if type(None) in target_type.__args__:\n optional_type = target_type.__args__[0]\n return _decode(value, optional_type)\n\n # List[T]\n if isinstance(value, Sequence):\n list_type = target_type.__args__[0]\n return [_decode(item, list_type) for item in value]\n\n # Dict[str, T]\n if isinstance(value, Mapping):\n value_type = target_type.__args__[1]\n return {\n map_key: _decode(map_value, value_type)\n for map_key, map_value in value.items()\n }\n\n return value\n" }, { "path": "rhasspy3/util/jaml.py", "content": "\"\"\"JAML is JSON objects as a *severely* restricted subset of YAML.\"\"\"\nfrom collections.abc import Mapping\nfrom enum import Enum, auto\nfrom typing import IO, Any, Dict, List, Union\n\n_INDENT = 2\n\n\ndef safe_load(fp: IO[str]) -> Dict[str, Any]:\n loader = JamlLoader()\n for line in fp:\n loader.process_line(line)\n\n return loader.output\n\n\nclass LoaderState(Enum):\n IN_DICT = auto()\n LITERAL = auto()\n\n\nclass JamlLoader:\n def __init__(self) -> None:\n self.output: Dict[str, Any] = {}\n self.indent = 0\n self.state = LoaderState.IN_DICT\n self.literal = \"\"\n self.target_stack: List[Union[Dict[str, Any], str]] = [self.output]\n\n def process_line(self, line: str):\n line_stripped = line.strip()\n if line_stripped.startswith(\"#\") or (not line_stripped):\n # Comment or empty line\n return\n\n line_indent = len(line) - len(line.lstrip())\n if self.state == LoaderState.LITERAL:\n # Multi-line literal\n if line_indent < self.indent:\n # Done with literal\n assert len(self.target_stack) > 1\n key = self.target_stack.pop()\n assert isinstance(key, str)\n\n target = self.target_stack[-1]\n assert isinstance(target, Mapping)\n target[key] = self.literal.strip()\n\n # Reset indent and state\n self.indent -= _INDENT\n self.state = LoaderState.IN_DICT\n else:\n # Add to literal\n self.literal += \"\\n\" + line.strip()\n\n if self.state == LoaderState.IN_DICT:\n self._add_key(line, line_indent)\n\n def _add_key(self, line, line_indent: int):\n while line_indent < self.indent:\n self.target_stack.pop()\n self.indent -= _INDENT\n\n assert self.target_stack\n target = self.target_stack[-1]\n assert isinstance(target, Mapping)\n\n parts = line.split(\":\", maxsplit=1)\n assert len(parts) == 2\n key = parts[0].strip()\n value = parts[1].strip()\n\n assert not key.startswith(\"-\"), \"Lists are not supported\"\n\n # Remove inline comments\n if value and (value[0] in (\"'\", '\"')):\n # Just keep what's in quotes.\n # This doesn't take escapes, etc. into account.\n end_quote = value.find(value[0], 1)\n value = value[: end_quote + 1]\n else:\n # Remove comment\n value = value.split(\"#\", maxsplit=1)[0]\n\n value_is_dict = True\n\n if value:\n value_is_dict = False\n\n if value[0] in (\"'\", '\"'):\n # Remove quotes\n value = value[1:-1]\n elif value == \"|\":\n self.literal = \"\"\n self.target_stack.append(key)\n self.indent += _INDENT\n self.state = LoaderState.LITERAL\n return\n elif value.lower() in (\"true\", \"false\"):\n value = value.lower() == \"true\"\n else:\n try:\n value = int(value)\n except ValueError:\n try:\n value = float(value)\n except ValueError:\n pass\n\n if value_is_dict:\n new_target: Dict[str, Any] = {}\n target[key] = new_target\n self.target_stack.append(new_target)\n self.indent += _INDENT\n else:\n target[key] = value\n" }, { "path": "rhasspy3/vad.py", "content": "\"\"\"Voice activity detection.\"\"\"\nimport asyncio\nimport logging\nimport time\nfrom dataclasses import dataclass\nfrom typing import Iterable, Optional, Union\n\nfrom .audio import AudioChunk, AudioStop\nfrom .config import PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event, async_write_event\nfrom .program import create_process\n\nDOMAIN = \"vad\"\n_STARTED_TYPE = \"voice-started\"\n_STOPPED_TYPE = \"voice-stopped\"\n\n_LOGGER = logging.getLogger(__name__)\n\n\n@dataclass\nclass VoiceStarted(Eventable):\n \"\"\"User has started speaking.\"\"\"\n\n timestamp: Optional[int] = None\n \"\"\"Milliseconds\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _STARTED_TYPE\n\n def event(self) -> Event:\n return Event(\n type=_STARTED_TYPE,\n data={\"timestamp\": self.timestamp},\n )\n\n @staticmethod\n def from_event(event: Event) -> \"VoiceStarted\":\n return VoiceStarted(timestamp=event.data.get(\"timestamp\"))\n\n\n@dataclass\nclass VoiceStopped(Eventable):\n \"\"\"User has stopped speaking.\"\"\"\n\n timestamp: Optional[int] = None\n \"\"\"Milliseconds\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _STOPPED_TYPE\n\n def event(self) -> Event:\n return Event(\n type=_STOPPED_TYPE,\n data={\"timestamp\": self.timestamp},\n )\n\n @staticmethod\n def from_event(event: Event) -> \"VoiceStopped\":\n return VoiceStopped(timestamp=event.data.get(\"timestamp\"))\n\n\n@dataclass\nclass Segmenter:\n \"\"\"Segments an audio stream by speech.\"\"\"\n\n speech_seconds: float\n \"\"\"Seconds of speech before voice command has started.\"\"\"\n\n silence_seconds: float\n \"\"\"Seconds of silence after voice command has ended.\"\"\"\n\n timeout_seconds: float\n \"\"\"Maximum number of seconds before stopping with timeout=True.\"\"\"\n\n reset_seconds: float\n \"\"\"Seconds before reset start/stop time counters.\"\"\"\n\n started: bool = False\n \"\"\"True if user has started speaking\"\"\"\n\n start_timestamp: Optional[int] = None\n \"\"\"Timestamp when user started speaking.\"\"\"\n\n stopped: bool = False\n \"\"\"True if user has stopped speaking\"\"\"\n\n stop_timestamp: Optional[int] = None\n \"\"\"Timestamp when user stopped speaking.\"\"\"\n\n timeout: bool = False\n \"\"\"True if stopping was due to timeout.\"\"\"\n\n _in_command: bool = False\n \"\"\"True if inside voice command.\"\"\"\n\n _speech_seconds_left: float = 0.0\n \"\"\"Seconds left before considering voice command as started.\"\"\"\n\n _silence_seconds_left: float = 0.0\n \"\"\"Seconds left before considering voice command as stopped.\"\"\"\n\n _timeout_seconds_left: float = 0.0\n \"\"\"Seconds left before considering voice command timed out.\"\"\"\n\n _reset_seconds_left: float = 0.0\n \"\"\"Seconds left before resetting start/stop time counters.\"\"\"\n\n def __post_init__(self):\n self.reset()\n\n def reset(self):\n \"\"\"Resets all counters and state.\"\"\"\n self._speech_seconds_left = self.speech_seconds\n self._silence_seconds_left = self.silence_seconds\n self._timeout_seconds_left = self.timeout_seconds\n self._reset_seconds_left = self.reset_seconds\n self._in_command = False\n self.start_timestamp = None\n self.stop_timestamp = None\n\n def process(\n self, chunk: bytes, chunk_seconds: float, is_speech: bool, timestamp: int\n ):\n \"\"\"Process a single chunk of audio.\"\"\"\n self._timeout_seconds_left -= chunk_seconds\n if self._timeout_seconds_left <= 0:\n self.stop_timestamp = timestamp\n self.timeout = True\n self.stopped = True\n return\n\n if not self._in_command:\n if is_speech:\n self._reset_seconds_left = self.reset_seconds\n\n if self.start_timestamp is None:\n self.start_timestamp = timestamp\n\n self._speech_seconds_left -= chunk_seconds\n if self._speech_seconds_left <= 0:\n # Inside voice command\n self._in_command = True\n self.started = True\n else:\n # Reset if enough silence\n self._reset_seconds_left -= chunk_seconds\n if self._reset_seconds_left <= 0:\n self._speech_seconds_left = self.speech_seconds\n self.start_timestamp = None\n else:\n if not is_speech:\n self._reset_seconds_left = self.reset_seconds\n self._silence_seconds_left -= chunk_seconds\n if self._silence_seconds_left <= 0:\n self.stop_timestamp = timestamp\n self.stopped = True\n else:\n # Reset if enough speech\n self._reset_seconds_left -= chunk_seconds\n if self._reset_seconds_left <= 0:\n self._silence_seconds_left = self.silence_seconds\n\n\nasync def segment(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n mic_in: asyncio.StreamReader,\n asr_out: asyncio.StreamWriter,\n chunk_buffer: Optional[Iterable[Event]] = None,\n):\n \"\"\"Segments an audio input stream, passing audio chunks to asr.\"\"\"\n async with (await create_process(rhasspy, DOMAIN, program)) as vad_proc:\n assert vad_proc.stdin is not None\n assert vad_proc.stdout is not None\n\n if chunk_buffer:\n # Buffered chunks from wake word detection\n for buffered_event in chunk_buffer:\n await asyncio.gather(\n async_write_event(buffered_event, vad_proc.stdin),\n async_write_event(buffered_event, asr_out),\n )\n\n mic_task = asyncio.create_task(async_read_event(mic_in))\n vad_task = asyncio.create_task(async_read_event(vad_proc.stdout))\n pending = {mic_task, vad_task}\n\n timestamp = 0\n in_command = False\n is_first_chunk = True\n\n while True:\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n if mic_task in done:\n mic_event = mic_task.result()\n if mic_event is None:\n break\n\n # Process chunk\n if AudioChunk.is_type(mic_event.type):\n if is_first_chunk:\n is_first_chunk = False\n _LOGGER.debug(\"segment: processing audio\")\n\n chunk = AudioChunk.from_event(mic_event)\n timestamp = (\n chunk.timestamp\n if chunk.timestamp is not None\n else time.monotonic_ns()\n )\n\n if in_command:\n # Speech recognition and silence detection\n await asyncio.gather(\n async_write_event(mic_event, asr_out),\n async_write_event(mic_event, vad_proc.stdin),\n )\n else:\n # Voice detection\n await asyncio.gather(\n async_write_event(mic_event, asr_out),\n async_write_event(mic_event, vad_proc.stdin),\n )\n\n # Next chunk\n mic_task = asyncio.create_task(async_read_event(mic_in))\n pending.add(mic_task)\n\n if vad_task in done:\n vad_event = vad_task.result()\n if vad_event is None:\n break\n\n if VoiceStarted.is_type(vad_event.type):\n if not in_command:\n # Start of voice command\n in_command = True\n _LOGGER.debug(\"segment: speaking started\")\n elif VoiceStopped.is_type(vad_event.type):\n # End of voice command\n _LOGGER.debug(\"segment: speaking ended\")\n await async_write_event(\n AudioStop(timestamp=timestamp).event(), asr_out\n )\n break\n\n # Next VAD event\n vad_task = asyncio.create_task(async_read_event(vad_proc.stdout))\n pending.add(vad_task)\n" }, { "path": "rhasspy3/wake.py", "content": "\"\"\"Wake word detection\"\"\"\nimport asyncio\nimport logging\nfrom dataclasses import dataclass\nfrom typing import AsyncIterable, MutableSequence, Optional, Union\n\nfrom .audio import AudioChunk, AudioStart, AudioStop\nfrom .config import PipelineProgramConfig\nfrom .core import Rhasspy\nfrom .event import Event, Eventable, async_read_event, async_write_event\nfrom .program import create_process\n\nDOMAIN = \"wake\"\n_DETECTION_TYPE = \"detection\"\n_NOT_DETECTED_TYPE = \"not-detected\"\n\n_LOGGER = logging.getLogger(__name__)\n\n\n@dataclass\nclass Detection(Eventable):\n \"\"\"Wake word was detected.\"\"\"\n\n name: Optional[str] = None\n \"\"\"Name of model.\"\"\"\n\n timestamp: Optional[int] = None\n \"\"\"Timestamp of audio chunk with detection\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _DETECTION_TYPE\n\n def event(self) -> Event:\n return Event(\n type=_DETECTION_TYPE, data={\"name\": self.name, \"timestamp\": self.timestamp}\n )\n\n @staticmethod\n def from_event(event: Event) -> \"Detection\":\n assert event.data is not None\n return Detection(\n name=event.data.get(\"name\"), timestamp=event.data.get(\"timestamp\")\n )\n\n\n@dataclass\nclass NotDetected(Eventable):\n \"\"\"Audio stream ended before wake word was detected.\"\"\"\n\n @staticmethod\n def is_type(event_type: str) -> bool:\n return event_type == _NOT_DETECTED_TYPE\n\n def event(self) -> Event:\n return Event(type=_NOT_DETECTED_TYPE)\n\n @staticmethod\n def from_event(event: Event) -> \"NotDetected\":\n return NotDetected()\n\n\nasync def detect(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n mic_in: asyncio.StreamReader,\n chunk_buffer: Optional[MutableSequence[Event]] = None,\n) -> Optional[Detection]:\n \"\"\"Try to detect wake word in an audio stream.\"\"\"\n detection: Optional[Detection] = None\n async with (await create_process(rhasspy, DOMAIN, program)) as wake_proc:\n assert wake_proc.stdin is not None\n assert wake_proc.stdout is not None\n\n mic_task = asyncio.create_task(async_read_event(mic_in))\n wake_task = asyncio.create_task(async_read_event(wake_proc.stdout))\n pending = {mic_task, wake_task}\n is_first_chunk = True\n\n while True:\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n if mic_task in done:\n mic_event = mic_task.result()\n if mic_event is None:\n break\n\n if AudioChunk.is_type(mic_event.type):\n if is_first_chunk:\n is_first_chunk = False\n _LOGGER.debug(\"detect: processing audio\")\n\n await async_write_event(mic_event, wake_proc.stdin)\n if chunk_buffer is not None:\n # Buffer chunks for asr\n chunk_buffer.append(mic_event)\n\n if detection is None:\n # Next chunk\n mic_task = asyncio.create_task(async_read_event(mic_in))\n pending.add(mic_task)\n\n if detection is not None:\n # Ensure last mic task is finished\n break\n\n if wake_task in done:\n wake_event = wake_task.result()\n if wake_event is None:\n break\n\n if Detection.is_type(wake_event.type):\n detection = Detection.from_event(wake_event)\n else:\n # Next wake event\n wake_task = asyncio.create_task(async_read_event(wake_proc.stdout))\n pending.add(wake_task)\n\n _LOGGER.debug(\"detect: %s\", detection)\n\n return detection\n\n\nasync def detect_stream(\n rhasspy: Rhasspy,\n program: Union[str, PipelineProgramConfig],\n audio_stream: AsyncIterable[bytes],\n rate: int,\n width: int,\n channels: int,\n) -> Optional[Detection]:\n \"\"\"Try to detect the wake word in a raw audio stream.\"\"\"\n async with (await create_process(rhasspy, DOMAIN, program)) as wake_proc:\n assert wake_proc.stdin is not None\n assert wake_proc.stdout is not None\n\n timestamp = 0\n await async_write_event(\n AudioStart(rate, width, channels, timestamp=timestamp).event(),\n wake_proc.stdin,\n )\n\n async def next_chunk():\n \"\"\"Get the next chunk from audio stream.\"\"\"\n async for chunk_bytes in audio_stream:\n return chunk_bytes\n\n audio_task = asyncio.create_task(next_chunk())\n wake_task = asyncio.create_task(async_read_event(wake_proc.stdout))\n pending = {audio_task, wake_task}\n\n while True:\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n if audio_task in done:\n chunk_bytes = audio_task.result()\n if chunk_bytes:\n chunk = AudioChunk(rate, width, channels, chunk_bytes)\n await async_write_event(chunk.event(), wake_proc.stdin)\n timestamp += chunk.milliseconds\n\n audio_task = asyncio.create_task(next_chunk())\n pending.add(audio_task)\n else:\n wake_task.cancel()\n await async_write_event(AudioStop().event(), wake_proc.stdin)\n wake_task = asyncio.create_task(async_read_event(wake_proc.stdout))\n pending = {wake_task}\n\n if wake_task in done:\n wake_event = wake_task.result()\n if wake_event is None:\n break\n\n if Detection.is_type(wake_event.type):\n detection = Detection.from_event(wake_event)\n _LOGGER.debug(\"detect: %s\", detection)\n return detection\n\n if NotDetected.is_type(wake_event.type):\n break\n\n wake_task = asyncio.create_task(async_read_event(wake_proc.stdout))\n pending.add(wake_task)\n\n _LOGGER.debug(\"Not detected\")\n\n return None\n" }, { "path": "rhasspy3_http_api/__init__.py", "content": "" }, { "path": "rhasspy3_http_api/__main__.py", "content": "import argparse\nimport asyncio\nimport logging\nimport os\nimport subprocess\nimport threading\nfrom pathlib import Path\nfrom typing import Tuple\nfrom uuid import uuid4\n\nimport hypercorn\nimport quart_cors\nfrom quart import Quart, Response, jsonify, render_template, send_from_directory\n\nfrom rhasspy3.audio import DEFAULT_SAMPLES_PER_CHUNK\nfrom rhasspy3.core import Rhasspy\n\nfrom .asr import add_asr\nfrom .handle import add_handle\nfrom .intent import add_intent\nfrom .pipeline import add_pipeline\nfrom .snd import add_snd\nfrom .tts import add_tts\nfrom .wake import add_wake\n\n_DIR = Path(__file__).parent\n_LOGGER = logging.getLogger(\"rhasspy\")\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument(\n \"-c\",\n \"--config\",\n default=_DIR.parent / \"config\",\n help=\"Configuration directory\",\n )\n parser.add_argument(\n \"--pipeline\", default=\"default\", help=\"Name of default pipeline to run\"\n )\n parser.add_argument(\n \"--server\",\n nargs=2,\n action=\"append\",\n metavar=(\"domain\", \"name\"),\n help=\"Domain/name of server(s) to run\",\n )\n parser.add_argument(\n \"--host\", default=\"0.0.0.0\", help=\"Host of HTTP server (default: 0.0.0.0)\"\n )\n parser.add_argument(\n \"--port\", type=int, default=13331, help=\"Port of HTTP server (default: 13331)\"\n )\n parser.add_argument(\n \"--samples-per-chunk\", type=int, default=DEFAULT_SAMPLES_PER_CHUNK\n )\n parser.add_argument(\"--asr-chunks-to-buffer\", type=int, default=0)\n parser.add_argument(\"--debug\", action=\"store_true\", help=\"Log DEBUG messages\")\n args = parser.parse_args()\n logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)\n\n rhasspy = Rhasspy.load(args.config)\n pipeline = rhasspy.config.pipelines[args.pipeline]\n\n template_dir = _DIR / \"templates\"\n img_dir = _DIR / \"img\"\n css_dir = _DIR / \"css\"\n js_dir = _DIR / \"js\"\n\n app = Quart(\"rhasspy3\", template_folder=str(template_dir))\n app.secret_key = str(uuid4())\n\n # Monkey patch quart_cors to get rid of non-standard requirement that\n # websockets have origin header set.\n def _apply_websocket_cors(*args, **kwargs):\n \"\"\"Allow null origin.\"\"\"\n pass\n\n # pylint: disable=protected-access\n quart_cors._apply_websocket_cors = _apply_websocket_cors\n app = quart_cors.cors(app, allow_origin=\"*\")\n\n add_wake(app, rhasspy, pipeline, args)\n add_asr(app, rhasspy, pipeline, args)\n add_intent(app, rhasspy, pipeline, args)\n add_handle(app, rhasspy, pipeline, args)\n add_snd(app, rhasspy, pipeline, args)\n add_tts(app, rhasspy, pipeline, args)\n add_pipeline(app, rhasspy, pipeline, args)\n\n @app.errorhandler(Exception)\n async def handle_error(err) -> Tuple[str, int]:\n \"\"\"Return error as text.\"\"\"\n _LOGGER.exception(err)\n return (f\"{err.__class__.__name__}: {err}\", 500)\n\n @app.route(\"/\", methods=[\"GET\"])\n async def page_index() -> str:\n \"\"\"Render main web page.\"\"\"\n return await render_template(\"index.html\", config=rhasspy.config)\n\n @app.route(\"/satellite.html\", methods=[\"GET\"])\n async def page_satellite() -> str:\n \"\"\"Render satellite web page.\"\"\"\n return await render_template(\"satellite.html\", config=rhasspy.config)\n\n @app.route(\"/img/\", methods=[\"GET\"])\n async def img(filename) -> Response:\n \"\"\"Image static endpoint.\"\"\"\n return await send_from_directory(img_dir, filename)\n\n @app.route(\"/css/\", methods=[\"GET\"])\n async def css(filename) -> Response:\n \"\"\"css static endpoint.\"\"\"\n return await send_from_directory(css_dir, filename)\n\n @app.route(\"/js/\", methods=[\"GET\"])\n async def js(filename) -> Response:\n \"\"\"Javascript static endpoint.\"\"\"\n return await send_from_directory(js_dir, filename)\n\n @app.route(\"/config\", methods=[\"GET\"])\n async def http_config() -> Response:\n return jsonify(rhasspy.config)\n\n @app.route(\"/version\", methods=[\"POST\"])\n async def http_version() -> str:\n return \"3.0.0\"\n\n hyp_config = hypercorn.config.Config()\n hyp_config.bind = [f\"{args.host}:{args.port}\"]\n\n if args.server:\n run_servers(rhasspy, args.server)\n\n try:\n asyncio.run(hypercorn.asyncio.serve(app, hyp_config))\n except KeyboardInterrupt:\n pass\n\n\n# -----------------------------------------------------------------------------\n\n\ndef run_servers(rhasspy, servers):\n def run_server(domain: str, name: str):\n try:\n command = [\n \"server_run.py\",\n \"--config\",\n str(rhasspy.config_dir),\n domain,\n name,\n ]\n env = dict(os.environ)\n env[\"PATH\"] = f'{rhasspy.base_dir}/bin:{env[\"PATH\"]}'\n _LOGGER.debug(command)\n _LOGGER.info(\"Starting %s %s\", domain, name)\n subprocess.run(command, check=True, cwd=rhasspy.base_dir, env=env)\n except Exception:\n _LOGGER.exception(\n \"Unexpected error running server: domain=%s, name=%s\", domain, name\n )\n\n for domain, server_name in servers:\n threading.Thread(\n target=run_server, args=(domain, server_name), daemon=True\n ).start()\n\n\n# -----------------------------------------------------------------------------\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "rhasspy3_http_api/asr.py", "content": "import argparse\nimport io\nimport json\nimport logging\n\nfrom quart import Quart, Response, jsonify, render_template, request, websocket\n\nfrom rhasspy3.asr import transcribe, transcribe_stream\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n AudioStop,\n)\nfrom rhasspy3.config import PipelineConfig\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import Event\n\n_LOGGER = logging.getLogger(__name__)\n\n\ndef add_asr(\n app: Quart, rhasspy: Rhasspy, pipeline: PipelineConfig, args: argparse.Namespace\n) -> None:\n @app.route(\"/asr.html\", methods=[\"GET\"])\n async def http_asr() -> str:\n return await render_template(\"asr.html\", config=rhasspy.config)\n\n @app.route(\"/asr/transcribe\", methods=[\"POST\"])\n async def http_asr_transcribe() -> Response:\n \"\"\"Transcribe a WAV file.\"\"\"\n wav_bytes = await request.data\n asr_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n\n asr_program = request.args.get(\"asr_program\") or asr_pipeline.asr\n assert asr_program, \"Missing program for asr\"\n\n samples_per_chunk = int(\n request.args.get(\"samples_per_chunk\", args.samples_per_chunk)\n )\n\n _LOGGER.debug(\"transcribe: asr=%s, wav=%s byte(s)\", asr_program, len(wav_bytes))\n\n with io.BytesIO(wav_bytes) as wav_in:\n transcript = await transcribe(\n rhasspy, asr_program, wav_in, samples_per_chunk\n )\n\n _LOGGER.debug(\"transcribe: transcript='%s'\", transcript)\n return jsonify(transcript.event().to_dict() if transcript is not None else {})\n\n @app.websocket(\"/asr/transcribe\")\n async def ws_asr_transcribe():\n \"\"\"Transcribe a websocket audio stream.\"\"\"\n asr_pipeline = (\n rhasspy.config.pipelines[websocket.args[\"pipeline\"]]\n if \"pipeline\" in websocket.args\n else pipeline\n )\n asr_program = websocket.args.get(\"asr_program\") or asr_pipeline.asr\n assert asr_program, \"Missing program for asr\"\n\n vad_program = websocket.args.get(\"vad_program\") or asr_pipeline.vad\n assert vad_program, \"Missing program for vad\"\n\n rate = int(websocket.args.get(\"rate\", DEFAULT_IN_RATE))\n width = int(websocket.args.get(\"width\", DEFAULT_IN_WIDTH))\n channels = int(websocket.args.get(\"channels\", DEFAULT_IN_CHANNELS))\n\n _LOGGER.debug(\n \"transcribe: asr=%s, vad=%s, rate=%s, width=%s, channels=%s\",\n asr_program,\n vad_program,\n rate,\n width,\n channels,\n )\n\n async def audio_stream():\n while True:\n data = await websocket.receive()\n if not data:\n # Empty message signals stop\n break\n\n if isinstance(data, bytes):\n # Raw audio\n yield data\n else:\n event = Event.from_dict(json.loads(data))\n if AudioStop.is_type(event.type):\n # Stop event\n break\n\n transcript = await transcribe_stream(\n rhasspy, asr_program, vad_program, audio_stream(), rate, width, channels\n )\n\n _LOGGER.debug(\"transcribe: transcript='%s'\", transcript)\n\n await websocket.send_json(\n transcript.event().to_dict() if transcript is not None else {}\n )\n" }, { "path": "rhasspy3_http_api/css/main.css", "content": "#header {\n border-bottom: 1px solid black;\n}\n\nli {\n padding-bottom: 1rem;\n}\n" }, { "path": "rhasspy3_http_api/handle.py", "content": "import argparse\nimport json\nimport logging\nfrom typing import Optional, Union\n\nfrom quart import Quart, Response, jsonify, request\n\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.config import PipelineConfig\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import Event\nfrom rhasspy3.handle import handle\nfrom rhasspy3.intent import Intent, NotRecognized\n\n_LOGGER = logging.getLogger(__name__)\n_HANDLE_INPUT_TYPES = (Transcript, Intent, NotRecognized)\n\n\ndef add_handle(\n app: Quart, rhasspy: Rhasspy, pipeline: PipelineConfig, args: argparse.Namespace\n) -> None:\n @app.route(\"/handle/handle\", methods=[\"GET\", \"POST\"])\n async def http_handle_handle() -> Response:\n \"\"\"Handle text or intent JSON.\"\"\"\n if request.method == \"GET\":\n data = request.args[\"input\"]\n else:\n data = (await request.data).decode()\n\n handle_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n\n # Input can be plain text or a JSON intent\n handle_input: Optional[Union[Intent, NotRecognized, Transcript]] = None\n if request.content_type == \"application/json\":\n # Try to parse either an \"intent\" or \"not-recognized\" event\n event = Event(json.loads(data))\n for event_class in _HANDLE_INPUT_TYPES:\n assert issubclass(event_class, _HANDLE_INPUT_TYPES)\n if event_class.is_type(event.type):\n handle_input = event_class.from_event(event)\n else:\n # Assume plain text\n handle_input = Transcript(data)\n\n assert handle_input is not None, \"Invalid input\"\n\n handle_program = request.args.get(\"handle_program\") or handle_pipeline.handle\n assert handle_program is not None, \"Missing program for handle\"\n _LOGGER.debug(\"handle: handle=%s, input='%s'\", handle_program, handle_input)\n\n result = await handle(rhasspy, handle_program, handle_input)\n _LOGGER.debug(\"handle: result=%s\", result)\n\n return jsonify(result.event().to_dict() if result is not None else {})\n" }, { "path": "rhasspy3_http_api/intent.py", "content": "import argparse\nimport logging\n\nfrom quart import Quart, Response, jsonify, request\n\nfrom rhasspy3.config import PipelineConfig\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.intent import recognize\n\n_LOGGER = logging.getLogger(__name__)\n\n\ndef add_intent(\n app: Quart, rhasspy: Rhasspy, pipeline: PipelineConfig, args: argparse.Namespace\n) -> None:\n @app.route(\"/intent/recognize\", methods=[\"GET\", \"POST\"])\n async def http_intent_recognize() -> Response:\n \"\"\"Recognize intent from text.\"\"\"\n if request.method == \"GET\":\n text = request.args[\"text\"]\n else:\n text = (await request.data).decode()\n\n intent_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n intent_program = request.args.get(\"intent_program\") or intent_pipeline.intent\n assert intent_program, \"Missing program for intent\"\n _LOGGER.debug(\"recognize: intent=%s, text='%s'\", intent_program, text)\n\n result = await recognize(rhasspy, intent_program, text)\n _LOGGER.debug(\"recognize: result=%s\", result)\n\n return jsonify(result.event().to_dict() if result is not None else {})\n" }, { "path": "rhasspy3_http_api/js/main.js", "content": "function q(selector) {\n return document.querySelector(selector);\n}\n\n// https://stackoverflow.com/questions/62093473/how-to-play-raw-audio-files\nfunction buildWaveHeader(opts) {\n const numFrames = opts.numFrames;\n const numChannels = opts.numChannels || 2;\n const sampleRate = opts.sampleRate || 44100;\n const bytesPerSample = opts.bytesPerSample || 2;\n const format = opts.format\n\n const blockAlign = numChannels * bytesPerSample;\n const byteRate = sampleRate * blockAlign;\n const dataSize = numFrames * blockAlign;\n\n const buffer = new ArrayBuffer(44);\n const dv = new DataView(buffer);\n\n let p = 0;\n\n function writeString(s) {\n for (let i = 0; i < s.length; i++) {\n dv.setUint8(p + i, s.charCodeAt(i));\n }\n p += s.length;\n }\n\n function writeUint32(d) {\n dv.setUint32(p, d, true);\n p += 4;\n }\n\n function writeUint16(d) {\n dv.setUint16(p, d, true);\n p += 2;\n }\n\n writeString('RIFF'); // ChunkID\n writeUint32(dataSize + 36); // ChunkSize\n writeString('WAVE'); // Format\n writeString('fmt '); // Subchunk1ID\n writeUint32(16); // Subchunk1Size\n writeUint16(format); // AudioFormat\n writeUint16(numChannels); // NumChannels\n writeUint32(sampleRate); // SampleRate\n writeUint32(byteRate); // ByteRate\n writeUint16(blockAlign); // BlockAlign\n writeUint16(bytesPerSample * 8); // BitsPerSample\n writeString('data'); // Subchunk2ID\n writeUint32(dataSize); // Subchunk2Size\n\n return buffer;\n}\n" }, { "path": "rhasspy3_http_api/js/recorder.worklet.js", "content": "class RecorderProcessor extends AudioWorkletProcessor {\n constructor() {\n super();\n }\n\n process(inputList, outputList, parameters) {\n if (inputList[0].length < 1) {\n return true;\n }\n\n const float32Data = inputList[0][0];\n const int16Data = new Int16Array(float32Data.length);\n\n for (let i = 0; i < float32Data.length; i++) {\n const s = Math.max(-1, Math.min(1, float32Data[i]));\n int16Data[i] = s < 0 ? s * 0x8000 : s * 0x7fff;\n }\n\n this.port.postMessage(int16Data);\n\n return true;\n }\n};\n\nregisterProcessor(\"recorder.worklet\", RecorderProcessor);\n" }, { "path": "rhasspy3_http_api/pipeline.py", "content": "import argparse\nimport asyncio\nimport io\nimport logging\nfrom enum import Enum\nfrom typing import IO, Optional, Union\n\nfrom quart import Quart, Response, jsonify, render_template, request, websocket\n\nfrom rhasspy3.asr import DOMAIN as ASR_DOMAIN\nfrom rhasspy3.asr import Transcript\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n DEFAULT_OUT_CHANNELS,\n DEFAULT_OUT_RATE,\n DEFAULT_OUT_WIDTH,\n AudioChunk,\n AudioChunkConverter,\n AudioStart,\n AudioStop,\n)\nfrom rhasspy3.config import PipelineConfig\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import Event, async_read_event, async_write_event\nfrom rhasspy3.handle import Handled, NotHandled, handle\nfrom rhasspy3.intent import Intent, NotRecognized\nfrom rhasspy3.pipeline import StopAfterDomain\nfrom rhasspy3.pipeline import run as run_pipeline\nfrom rhasspy3.program import create_process\nfrom rhasspy3.tts import synthesize_stream\nfrom rhasspy3.vad import DOMAIN as VAD_DOMAIN\nfrom rhasspy3.vad import VoiceStarted, VoiceStopped\nfrom rhasspy3.wake import Detection\n\n_LOGGER = logging.getLogger(__name__)\n\n\nclass StartAfterDomain(str, Enum):\n WAKE = \"wake\"\n ASR = \"asr\"\n INTENT = \"intent\"\n HANDLE = \"handle\"\n TTS = \"tts\"\n\n\ndef add_pipeline(\n app: Quart, rhasspy: Rhasspy, pipeline: PipelineConfig, args: argparse.Namespace\n) -> None:\n @app.route(\"/pipeline.html\", methods=[\"GET\"])\n async def http_pipeline() -> str:\n return await render_template(\"pipeline.html\", config=rhasspy.config)\n\n @app.route(\"/pipeline/run\", methods=[\"POST\"])\n async def http_pipeline_run() -> Response:\n running_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n mic_program = request.args.get(\"mic_program\") or running_pipeline.mic\n wake_program = request.args.get(\"wake_program\") or running_pipeline.wake\n vad_program = request.args.get(\"vad_program\") or running_pipeline.vad\n asr_program = request.args.get(\"asr_program\") or running_pipeline.asr\n intent_program = request.args.get(\"intent_program\") or running_pipeline.intent\n handle_program = request.args.get(\"handle_program\") or running_pipeline.handle\n tts_program = request.args.get(\"tts_program\") or running_pipeline.tts\n snd_program = request.args.get(\"snd_program\") or running_pipeline.snd\n #\n start_after = request.args.get(\"start_after\")\n stop_after = request.args.get(\"stop_after\")\n #\n samples_per_chunk = int(\n request.args.get(\"samples_per_chunk\", args.samples_per_chunk)\n )\n asr_chunks_to_buffer = int(\n request.args.get(\"asr_chunks_to_buffer\", args.asr_chunks_to_buffer)\n )\n\n _LOGGER.debug(\n \"run: \"\n \"mic=%s,\"\n \"wake=%s,\"\n \"vad=%s,\"\n \"asr=%s,\"\n \"intent=%s,\"\n \"handle=%s,\"\n \"tts=%s,\"\n \"snd=%s,\"\n \"start_after=%s \"\n \"stop_after=%s\",\n mic_program,\n wake_program,\n vad_program,\n asr_program,\n intent_program,\n handle_program,\n tts_program,\n snd_program,\n start_after,\n stop_after,\n )\n\n wake_detection: Optional[Detection] = None\n asr_wav_in: Optional[IO[bytes]] = None\n asr_transcript: Optional[Transcript] = None\n intent_result: Optional[Union[Intent, NotRecognized]] = None\n handle_result: Optional[Union[Handled, NotHandled]] = None\n tts_wav_in: Optional[IO[bytes]] = None\n\n if start_after:\n # Determine where to start in the pipeline\n start_after = StartAfterDomain(start_after)\n if start_after == StartAfterDomain.WAKE:\n # Body is detected wake name\n name = (await request.data).decode()\n wake_detection = Detection(name=name)\n elif start_after == StartAfterDomain.ASR:\n # Body is transcript or WAV\n if request.content_type == \"audio/wav\":\n wav_bytes = await request.data\n asr_wav_in = io.BytesIO(wav_bytes)\n else:\n text = (await request.data).decode()\n asr_transcript = Transcript(text=text)\n elif start_after == StartAfterDomain.INTENT:\n # Body is JSON\n event = Event.from_dict(await request.json)\n if Intent.is_type(event.type):\n intent_result = Intent.from_event(event)\n elif NotRecognized.is_type(event.type):\n intent_result = NotRecognized.from_event(event)\n else:\n raise ValueError(f\"Unexpected event type: {event.type}\")\n elif start_after == StartAfterDomain.HANDLE:\n # Body is text or JSON\n if request.content_type == \"application/json\":\n event = Event.from_dict(await request.json)\n if Handled.is_type(event.type):\n handle_result = Handled.from_event(event)\n elif NotRecognized.is_type(event.type):\n handle_result = NotHandled.from_event(event)\n else:\n raise ValueError(f\"Unexpected event type: {event.type}\")\n else:\n # Plain text\n text = (await request.data).decode()\n handle_result = Handled(text=text)\n elif start_after == StartAfterDomain.TTS:\n # Body is or WAV\n wav_bytes = await request.data\n tts_wav_in = io.BytesIO(wav_bytes)\n\n pipeline_result = await run_pipeline(\n rhasspy,\n pipeline,\n samples_per_chunk,\n asr_chunks_to_buffer=asr_chunks_to_buffer,\n mic_program=mic_program,\n wake_program=wake_program,\n wake_detection=wake_detection,\n asr_program=asr_program,\n asr_transcript=asr_transcript,\n asr_wav_in=asr_wav_in,\n vad_program=vad_program,\n intent_program=intent_program,\n intent_result=intent_result,\n handle_program=handle_program,\n handle_result=handle_result,\n tts_program=tts_program,\n tts_wav_in=tts_wav_in,\n snd_program=snd_program,\n stop_after=StopAfterDomain(stop_after) if stop_after else None,\n )\n\n return jsonify(pipeline_result.to_event_dict())\n\n @app.websocket(\"/pipeline/asr-tts\")\n async def ws_api_asr_tts() -> None:\n running_pipeline = (\n rhasspy.config.pipelines[websocket.args[\"pipeline\"]]\n if \"pipeline\" in websocket.args\n else pipeline\n )\n\n asr_program = websocket.args.get(\"asr_program\") or running_pipeline.asr\n assert asr_program, \"Missing asr program\"\n\n vad_program = websocket.args.get(\"vad_program\") or running_pipeline.vad\n assert vad_program, \"Missing vad program\"\n\n handle_program = websocket.args.get(\"handle_program\") or running_pipeline.handle\n assert handle_program, \"Missing handle program\"\n\n tts_program = websocket.args.get(\"tts_program\") or running_pipeline.tts\n assert tts_program, \"Missing tts program\"\n\n in_rate = int(websocket.args.get(\"in_rate\", DEFAULT_IN_RATE))\n in_width = int(websocket.args.get(\"in_width\", DEFAULT_IN_WIDTH))\n in_channels = int(websocket.args.get(\"in_channels\", DEFAULT_IN_CHANNELS))\n\n out_rate = int(websocket.args.get(\"out_rate\", DEFAULT_OUT_RATE))\n out_width = int(websocket.args.get(\"out_width\", DEFAULT_OUT_WIDTH))\n out_channels = int(websocket.args.get(\"out_channels\", DEFAULT_OUT_CHANNELS))\n\n # asr + vad\n async with (\n await create_process(rhasspy, ASR_DOMAIN, asr_program)\n ) as asr_proc, (\n await create_process(rhasspy, VAD_DOMAIN, vad_program)\n ) as vad_proc:\n assert asr_proc.stdin is not None\n assert asr_proc.stdout is not None\n assert vad_proc.stdin is not None\n assert vad_proc.stdout is not None\n\n mic_task = asyncio.create_task(websocket.receive())\n vad_task = asyncio.create_task(async_read_event(vad_proc.stdout))\n pending = {mic_task, vad_task}\n\n while True:\n done, pending = await asyncio.wait(\n pending, return_when=asyncio.FIRST_COMPLETED\n )\n\n if mic_task in done:\n audio_bytes = mic_task.result()\n if isinstance(audio_bytes, bytes) and audio_bytes:\n mic_chunk = AudioChunk(\n in_rate, in_width, in_channels, audio_bytes\n )\n mic_chunk_event = mic_chunk.event()\n await asyncio.gather(\n async_write_event(mic_chunk_event, asr_proc.stdin),\n async_write_event(mic_chunk_event, vad_proc.stdin),\n )\n\n mic_task = asyncio.create_task(websocket.receive())\n pending.add(mic_task)\n\n if vad_task in done:\n vad_event = vad_task.result()\n if vad_event is None:\n break\n\n # Forward to websocket\n await websocket.send_json(vad_event.to_dict())\n\n if VoiceStarted.is_type(vad_event.type):\n _LOGGER.debug(\"stream-to-stream: voice started\")\n elif VoiceStopped.is_type(vad_event.type):\n _LOGGER.debug(\"stream-to-stream: voice stopped\")\n break\n\n vad_task = asyncio.create_task(async_read_event(vad_proc.stdout))\n pending.add(vad_task)\n\n # Get transcript from asr\n await async_write_event(AudioStop().event(), asr_proc.stdin)\n transcript: Optional[Transcript] = None\n while True:\n asr_event = await async_read_event(asr_proc.stdout)\n if asr_event is None:\n break\n\n # Forward to websocket\n await websocket.send_json(asr_event.to_dict())\n\n if Transcript.is_type(asr_event.type):\n transcript = Transcript.from_event(asr_event)\n _LOGGER.debug(\"stream-to-stream: asr=%s\", transcript)\n break\n\n handle_result: Optional[Union[Handled, NotHandled]] = None\n if transcript is not None:\n handle_result = await handle(rhasspy, handle_program, transcript)\n _LOGGER.debug(\"stream-to-stream: handle=%s\", handle_result)\n\n if (handle_result is not None) and handle_result.text:\n # Forward to websocket\n await websocket.send_json(handle_result.event().to_dict())\n\n _LOGGER.debug(\"stream-to-stream: sending tts\")\n await websocket.send_json(\n AudioStart(out_rate, out_width, out_channels).event().to_dict()\n )\n converter = AudioChunkConverter(out_rate, out_width, out_channels)\n async for tts_chunk in synthesize_stream(\n rhasspy, tts_program, handle_result.text\n ):\n tts_chunk = converter.convert(tts_chunk)\n await websocket.send(tts_chunk.audio)\n\n _LOGGER.debug(\"stream-to-stream: tts done\")\n\n await websocket.send_json(AudioStop().event().to_dict())\n" }, { "path": "rhasspy3_http_api/snd.py", "content": "import argparse\nimport io\nimport json\nimport logging\n\nfrom quart import Quart, Response, jsonify, request, websocket\n\nfrom rhasspy3.audio import (\n DEFAULT_OUT_CHANNELS,\n DEFAULT_OUT_RATE,\n DEFAULT_OUT_WIDTH,\n AudioStop,\n)\nfrom rhasspy3.config import PipelineConfig\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import Event\nfrom rhasspy3.snd import play, play_stream\n\n_LOGGER = logging.getLogger(__name__)\n\n\ndef add_snd(\n app: Quart, rhasspy: Rhasspy, pipeline: PipelineConfig, args: argparse.Namespace\n) -> None:\n @app.route(\"/snd/play\", methods=[\"POST\"])\n async def http_snd_play() -> Response:\n \"\"\"Play WAV file.\"\"\"\n wav_bytes = await request.data\n snd_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n snd_program = request.args.get(\"snd_program\") or snd_pipeline.snd\n assert snd_program, \"Missing program for snd\"\n\n samples_per_chunk = int(\n request.args.get(\"samples_per_chunk\", args.samples_per_chunk)\n )\n\n _LOGGER.debug(\"play: snd=%s, wav=%s byte(s)\", snd_program, len(wav_bytes))\n\n with io.BytesIO(wav_bytes) as wav_in:\n played = await play(\n rhasspy,\n snd_program,\n wav_in,\n samples_per_chunk,\n )\n\n return jsonify(played.event().to_dict() if played is not None else {})\n\n @app.websocket(\"/snd/play\")\n async def ws_snd_play():\n \"\"\"Play websocket audio stream.\"\"\"\n snd_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n snd_program = websocket.args.get(\"snd_program\") or snd_pipeline.snd\n assert snd_program, \"Missing program for snd\"\n\n rate = int(websocket.args.get(\"rate\", DEFAULT_OUT_RATE))\n width = int(websocket.args.get(\"width\", DEFAULT_OUT_WIDTH))\n channels = int(websocket.args.get(\"channels\", DEFAULT_OUT_CHANNELS))\n\n _LOGGER.debug(\"play: snd=%s\", snd_program)\n\n async def audio_stream():\n while True:\n data = await websocket.receive()\n if not data:\n # Empty message signals stop\n break\n\n if isinstance(data, bytes):\n # Raw audio\n yield data\n else:\n event = Event.from_dict(json.loads(data))\n if AudioStop.is_type(event.type):\n # Stop event\n break\n\n played = await play_stream(\n rhasspy, snd_program, audio_stream(), rate, width, channels\n )\n\n await websocket.send_json(\n played.event().to_dict() if played is not None else {}\n )\n" }, { "path": "rhasspy3_http_api/templates/asr.html", "content": "{% extends \"layout.html\" %}\n\n{% block body %}\n\n

Speech to Text (asr)

\n\n

Transcribe

\n\n
    \n
  1. \n Pipeline:\n \n
    2. \n
  2. \n Mic Program:\n \n \n
    3. \n \n
  3. \n VAD Program:\n \n \n
    4. \n
  4. \n ASR Program:\n \n \n
    5. \n
  5. \n \n
    6. \n
  6. \n Status: \n
    7. \n
  7. \n Transcript: \n
    8. \n
\n\n\n\n\n\n{% endblock %}\n" }, { "path": "rhasspy3_http_api/templates/index.html", "content": "{% extends \"layout.html\" %}\n\n{% block body %}\n\n\n\n{% endblock %}\n" }, { "path": "rhasspy3_http_api/templates/layout.html", "content": "\n\n \n \n \n \n \n \n Rhasspy\n \n \n
\n \n \n \n
\n\n {% block body %}{% endblock %}\n \n\n \n\n" }, { "path": "rhasspy3_http_api/templates/pipeline.html", "content": "{% extends \"layout.html\" %}\n\n{% block body %}\n\n

Pipeline

\n\n
    \n
  1. \n Pipeline:\n \n
    2. \n
  2. \n Mic Program:\n \n
    3. \n
  3. \n Wake Program:\n \n
    4. \n
  4. \n ASR Program:\n \n \n
    5. \n
  5. \n VAD Program:\n \n \n
    6. \n
  6. \n Intent Program:\n \n \n
    7. \n
  7. \n Handle Program:\n \n \n
    8. \n
  8. \n TTS Program:\n \n \n
    9. \n
  9. \n Snd Program:\n \n \n
    10. \n
  10. \n \n starting after\n \n stopping after\n \n
    11. \n
  11. \n Status: \n
    12. \n
  12. \n Result: \n
    13. \n
\n\n\n\n\n\n{% endblock %}\n" }, { "path": "rhasspy3_http_api/templates/satellite.html", "content": "{% extends \"layout.html\" %}\n\n{% block body %}\n\n

Satellite

\n\n
    \n
  1. \n Pipeline:\n \n
    2. \n
  2. \n ASR Program:\n \n \n
    3. \n
  3. \n VAD Program:\n \n \n
    4. \n
  4. \n Handle Program:\n \n \n
    5. \n
  5. \n TTS Program:\n \n \n
    6. \n
  6. \n \n
    7. \n
  7. \n Status: \n
    8. \n
  8. \n Events:\n
      \n
    \n
    9. \n
  9. \n \n
    10. \n
\n\n\n\n\n\n{% endblock %}\n" }, { "path": "rhasspy3_http_api/templates/tts.html", "content": "{% extends \"layout.html\" %}\n\n{% block body %}\n\n

Text to Speech (tts)

\n\n

Speak

\n\n
    \n
  1. \n Text: \n
    2. \n
  2. \n Pipeline:\n \n
    3. \n
  3. \n TTS Program:\n \n \n
    4. \n
  4. \n Snd Program:\n \n \n
    5. \n
  5. \n \n
    6. \n
  6. \n Status: \n
    7. \n
  7. \n \n
    8. \n
\n\n\n\n\n\n{% endblock %}\n" }, { "path": "rhasspy3_http_api/tts.py", "content": "import argparse\nimport io\nimport logging\n\nfrom quart import Quart, Response, jsonify, render_template, request\n\nfrom rhasspy3.config import PipelineConfig\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.snd import play\nfrom rhasspy3.tts import synthesize\n\n_LOGGER = logging.getLogger(__name__)\n\n\ndef add_tts(\n app: Quart, rhasspy: Rhasspy, pipeline: PipelineConfig, args: argparse.Namespace\n) -> None:\n @app.route(\"/tts.html\", methods=[\"GET\"])\n async def http_tts() -> str:\n return await render_template(\"tts.html\", config=rhasspy.config)\n\n @app.route(\"/tts/synthesize\", methods=[\"GET\", \"POST\"])\n async def http_tts_synthesize() -> Response:\n \"\"\"Synthesize a WAV file from text.\"\"\"\n if request.method == \"GET\":\n text = request.args[\"text\"]\n else:\n text = (await request.data).decode()\n\n tts_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n tts_program = request.args.get(\"tts_program\") or tts_pipeline.tts\n assert tts_program, \"No tts program\"\n _LOGGER.debug(\"synthesize: tts=%s, text='%s'\", tts_program, text)\n\n with io.BytesIO() as wav_out:\n await synthesize(rhasspy, tts_program, text, wav_out)\n wav_bytes = wav_out.getvalue()\n _LOGGER.debug(\"synthesize: wav=%s byte(s)\", len(wav_bytes))\n\n return Response(wav_bytes, mimetype=\"audio/wav\")\n\n @app.route(\"/tts/speak\", methods=[\"GET\", \"POST\"])\n async def http_tts_speak() -> Response:\n \"\"\"Synthesize audio from text and play.\"\"\"\n if request.method == \"GET\":\n text = request.args[\"text\"]\n else:\n text = (await request.data).decode()\n\n tts_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n tts_program = request.args.get(\"tts_program\") or tts_pipeline.tts\n snd_program = request.args.get(\"snd_program\") or tts_pipeline.snd\n samples_per_chunk = int(\n request.args.get(\"samples_per_chunk\", args.samples_per_chunk)\n )\n\n assert tts_program, \"No tts program\"\n assert snd_program, \"No snd program\"\n _LOGGER.debug(\n \"synthesize: tts=%s, snd=%s, text='%s'\", tts_program, snd_program, text\n )\n\n with io.BytesIO() as wav_out:\n await synthesize(rhasspy, tts_program, text, wav_out)\n wav_bytes = wav_out.getvalue()\n _LOGGER.debug(\"synthesize: wav=%s byte(s)\", len(wav_bytes))\n\n wav_out.seek(0)\n played = await play(rhasspy, snd_program, wav_out, samples_per_chunk)\n\n return jsonify(played.event().to_dict() if played is not None else {})\n" }, { "path": "rhasspy3_http_api/wake.py", "content": "import argparse\nimport io\nimport json\nimport logging\nimport wave\n\nfrom quart import Quart, Response, jsonify, request, websocket\n\nfrom rhasspy3.audio import (\n DEFAULT_IN_CHANNELS,\n DEFAULT_IN_RATE,\n DEFAULT_IN_WIDTH,\n AudioStop,\n)\nfrom rhasspy3.config import PipelineConfig\nfrom rhasspy3.core import Rhasspy\nfrom rhasspy3.event import Event\nfrom rhasspy3.mic import DOMAIN as MIC_DOMAIN\nfrom rhasspy3.program import create_process\nfrom rhasspy3.wake import detect, detect_stream\n\n_LOGGER = logging.getLogger(__name__)\n\n\ndef add_wake(\n app: Quart, rhasspy: Rhasspy, pipeline: PipelineConfig, args: argparse.Namespace\n) -> None:\n @app.route(\"/wake/detect\", methods=[\"GET\", \"POST\"])\n async def http_wake_detect() -> Response:\n \"\"\"Detect wake word in WAV file.\"\"\"\n wav_bytes = await request.data\n wake_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n wake_program = request.args.get(\"wake_program\") or wake_pipeline.wake\n assert wake_program, \"Missing program for wake\"\n\n if wav_bytes:\n # Detect from WAV\n samples_per_chunk = int(\n request.args.get(\"samples_per_chunk\", args.samples_per_chunk)\n )\n\n _LOGGER.debug(\n \"detect: wake=%s, wav=%s byte(s)\", wake_program, len(wav_bytes)\n )\n\n with io.BytesIO(wav_bytes) as wav_io:\n wav_file: wave.Wave_read = wave.open(wav_io, \"rb\")\n with wav_file:\n\n async def audio_stream():\n chunk = wav_file.readframes(samples_per_chunk)\n while chunk:\n yield chunk\n chunk = wav_file.readframes(samples_per_chunk)\n\n detection = await detect_stream(\n rhasspy,\n wake_program,\n audio_stream(),\n wav_file.getframerate(),\n wav_file.getsampwidth(),\n wav_file.getnchannels(),\n )\n else:\n # Detect from mic\n mic_program = request.args.get(\"mic_program\") or wake_pipeline.mic\n assert mic_program, \"Missing program for mic\"\n\n _LOGGER.debug(\"detect: mic=%s, wake=%s\", mic_program, wake_program)\n\n async with (\n await create_process(rhasspy, MIC_DOMAIN, mic_program)\n ) as mic_proc:\n assert mic_proc.stdout is not None\n detection = await detect(rhasspy, wake_program, mic_proc.stdout)\n\n _LOGGER.debug(\"wake: detection=%s\", detection)\n return jsonify(detection.event().to_dict() if detection is not None else {})\n\n @app.websocket(\"/wake/detect\")\n async def ws_wake_detect():\n \"\"\"Detect wake word in websocket audio stream.\"\"\"\n wake_pipeline = (\n rhasspy.config.pipelines[request.args[\"pipeline\"]]\n if \"pipeline\" in request.args\n else pipeline\n )\n wake_program = websocket.args.get(\"wake_program\") or wake_pipeline.wake\n assert wake_program, \"Missing program for wake\"\n\n rate = int(websocket.args.get(\"rate\", DEFAULT_IN_RATE))\n width = int(websocket.args.get(\"width\", DEFAULT_IN_WIDTH))\n channels = int(websocket.args.get(\"channels\", DEFAULT_IN_CHANNELS))\n\n _LOGGER.debug(\"detect: wake=%s\", wake_program)\n\n async def audio_stream():\n while True:\n data = await websocket.receive()\n if not data:\n # Empty message signals stop\n break\n\n if isinstance(data, bytes):\n # Raw audio\n yield data\n else:\n event = Event.from_dict(json.loads(data))\n if AudioStop.is_type(event.type):\n # Stop event\n break\n\n detection = await detect_stream(\n rhasspy, wake_program, audio_stream(), rate, width, channels\n )\n\n _LOGGER.debug(\"detect: detection='%s'\", detection)\n\n await websocket.send_json(\n detection.event().to_dict() if detection is not None else {}\n )\n" }, { "path": "script/format", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n # Activate virtual environment if available\n source \"${venv}/bin/activate\"\nfi\n\npython_files=()\npython_files+=(\"${base_dir}/bin\")\npython_files+=(\"${base_dir}/rhasspy3\")\npython_files+=(\"${base_dir}/rhasspy3_http_api\")\npython_files+=(\"${base_dir}/programs\")\n\n# Format code\nblack \"${python_files[@]}\"\nisort \"${python_files[@]}\"\n" }, { "path": "script/http_server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n # Activate virtual environment if available\n source \"${venv}/bin/activate\"\nfi\n\nexport PYTHONPATH=\"${base_dir}\"\nexport PATH=\"${base_dir}/bin:${PATH}\"\n\npython3 -m rhasspy3_http_api \"$@\"\n" }, { "path": "script/lint", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n # Activate virtual environment if available\n source \"${venv}/bin/activate\"\nfi\n\npython_files=()\npython_files+=(\"${base_dir}/bin\")\npython_files+=(\"${base_dir}/rhasspy3\")\npython_files+=(\"${base_dir}/rhasspy3_http_api\")\npython_files+=(\"${base_dir}/programs\")\n\n# Check\nblack \"${python_files[@]}\" --check\nisort \"${python_files[@]}\" --check\nflake8 \"${python_files[@]}\"\npylint \"${python_files[@]}\"\nmypy \"${base_dir}/bin\" \"${base_dir}/rhasspy3\"\n" }, { "path": "script/run", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n # Activate virtual environment if available\n source \"${venv}/bin/activate\"\nfi\n\nexport PYTHONPATH=\"${base_dir}\"\nexport PYTHONUNBUFFERED='1'\nexport PATH=\"${base_dir}/bin:${PATH}\"\n\n\"$@\"\n" }, { "path": "script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Copy default config\nconfig_dir=\"${base_dir}/config\"\n\nmkdir -p \"${config_dir}\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "script/setup_http_server", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\n# Create virtual environment\nif [ ! -d \"${venv}\" ]; then\n echo \"Creating virtual environment at ${venv} (${python_version})\"\n rm -rf \"${venv}\"\n \"${PYTHON}\" -m venv \"${venv}\"\n source \"${venv}/bin/activate\"\n\n # Install Python dependencies\n echo 'Installing Python dependencies'\n pip3 install --upgrade pip\n pip3 install --upgrade wheel setuptools\nelse\n source \"${venv}/bin/activate\"\nfi\n\nif [ -f \"${base_dir}/requirements.txt\" ]; then\n pip3 install -r \"${base_dir}/requirements.txt\"\nfi\n\npip3 install -r \"${base_dir}/requirements_http_api.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" }, { "path": "script/test", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n # Activate virtual environment if available\n source \"${venv}/bin/activate\"\nfi\n\nexport PYTHONPATH=\"${base_dir}:${PYTHONPATH}\"\npytest -vv \"${base_dir}/tests\"\n" }, { "path": "setup.cfg", "content": "[flake8]\n# To work with Black\nmax-line-length = 88\n# E501: line too long\n# W503: Line break occurred before a binary operator\n# E203: Whitespace before ':'\n# D202 No blank lines allowed after function docstring\n# W504 line break after binary operator\nignore =\n E501,\n W503,\n E203,\n D202,\n W504\n\n# F401 import unused\nper-file-ignores =\n programs/asr/faster-whisper/src/faster_whisper/__init__.py:F401\n\n[isort]\nmulti_line_output = 3\ninclude_trailing_comma=True\nforce_grid_wrap=0\nuse_parentheses=True\nline_length=88\nindent = \" \"\n" }, { "path": "setup.py", "content": "#!/usr/bin/env python3\nfrom pathlib import Path\n\nimport setuptools\nfrom setuptools import setup\n\nthis_dir = Path(__file__).parent\nmodule_dir = this_dir / \"rhasspy3\"\n\n# -----------------------------------------------------------------------------\n\n# Load README in as long description\nlong_description: str = \"\"\nreadme_path = this_dir / \"README.md\"\nif readme_path.is_file():\n long_description = readme_path.read_text(encoding=\"utf-8\")\n\nrequirements = []\nrequirements_path = this_dir / \"requirements.txt\"\nif requirements_path.is_file():\n with open(requirements_path, \"r\", encoding=\"utf-8\") as requirements_file:\n requirements = requirements_file.read().splitlines()\n\nversion_path = module_dir / \"VERSION\"\nwith open(version_path, \"r\", encoding=\"utf-8\") as version_file:\n version = version_file.read().strip()\n\n# -----------------------------------------------------------------------------\n\nsetup(\n name=\"rhasspy3\",\n version=version,\n description=\"Rhasspy Voice Assistant Toolkit\",\n long_description=long_description,\n url=\"http://github.com/rhasspy/rhasspy3\",\n author=\"Michael Hansen\",\n author_email=\"mike@rhasspy.org\",\n license=\"MIT\",\n packages=setuptools.find_packages(),\n package_data={\n \"rhasspy3\": [\"VERSION\", \"py.typed\"],\n },\n install_requires=requirements,\n classifiers=[\n \"Development Status :: 3 - Alpha\",\n \"Intended Audience :: Developers\",\n \"Topic :: Text Processing :: Linguistic\",\n \"License :: OSI Approved :: License :: OSI Approved :: MIT License\",\n \"Programming Language :: Python :: 3.7\",\n \"Programming Language :: Python :: 3.8\",\n \"Programming Language :: Python :: 3.9\",\n \"Programming Language :: Python :: 3.10\",\n ],\n keywords=\"voice assistant rhasspy\",\n)\n" }, { "path": "tests/test_dataclasses_json.py", "content": "from dataclasses import dataclass\nfrom typing import Dict, List, Optional\n\nfrom rhasspy3.util.dataclasses_json import DataClassJsonMixin\n\n\n@dataclass\nclass Class1(DataClassJsonMixin):\n name: str\n\n\n@dataclass\nclass Class2(DataClassJsonMixin):\n name: str\n obj1: Class1\n list1: List[Class1]\n dict1: Dict[str, Class1]\n opt1: Optional[Class1]\n\n\n_DICT = {\n \"name\": \"2\",\n \"obj1\": {\"name\": \"1\"},\n \"list1\": [{\"name\": \"1-2\"}],\n \"dict1\": {\"key\": {\"name\": \"1-3\"}},\n \"opt1\": {\"name\": \"1-4\"},\n}\n_OBJ = Class2(\n name=\"2\",\n obj1=Class1(name=\"1\"),\n list1=[Class1(name=\"1-2\")],\n dict1={\"key\": Class1(name=\"1-3\")},\n opt1=Class1(name=\"1-4\"),\n)\n\n\ndef test_to_dict():\n assert _OBJ.to_dict() == _DICT\n\n\ndef test_from_dict():\n assert Class2.from_dict(_DICT) == _OBJ\n" }, { "path": "tests/test_jaml.py", "content": "import io\n\nfrom rhasspy3.util.jaml import safe_load\n\nYAML = \"\"\"\n# Line comment\nouter_a: # Inline comment\n name: outer_a\n prop_int: 1\n prop_float: 1.23\n prop_bool: true\n prop_bool2: false\n prop_str_noquotes: hello: world\n prop_str_1quotes: 'hello: world'\n prop_str_2quotes: \"hello: world\"\n prop_str_literal: |\n hello:\n world\n inner_a:\n name: inner_a\n empty_string: \"\"\n string_with_hash: \"#test\"\n\nouter_b:\n name: inner_b\n\"\"\"\n\n\ndef test_safe_load():\n with io.StringIO(YAML) as yaml:\n assert safe_load(yaml) == {\n \"outer_a\": {\n \"name\": \"outer_a\",\n \"prop_int\": 1,\n \"prop_float\": 1.23,\n \"prop_bool\": True,\n \"prop_bool2\": False,\n \"prop_str_noquotes\": \"hello: world\",\n \"prop_str_1quotes\": \"hello: world\",\n \"prop_str_2quotes\": \"hello: world\",\n \"prop_str_literal\": \"hello:\\nworld\",\n \"inner_a\": {\"name\": \"inner_a\"},\n \"empty_string\": \"\",\n \"string_with_hash\": \"#test\",\n },\n \"outer_b\": {\"name\": \"inner_b\"},\n }\n" }, { "path": "tools/websocket-client/bin/websocket_client.py", "content": "#!/usr/bin/env python3\nimport argparse\nimport asyncio\nimport wave\nfrom urllib.parse import urlencode, urlparse, parse_qsl, urlunparse\n\nfrom websockets import connect\n\n\nasync def main() -> None:\n parser = argparse.ArgumentParser()\n parser.add_argument(\"url\")\n parser.add_argument(\"wav_file\", nargs=\"+\", help=\"Path(s) to WAV file(s)\")\n parser.add_argument(\"--samples-per-chunk\", type=int, default=1024)\n args = parser.parse_args()\n\n for wav_path in args.wav_file:\n wav_file: wave.Wave_read = wave.open(wav_path, \"rb\")\n with wav_file:\n # Add audio parameters if missing\n parse_result = urlparse(args.url)\n query = dict(parse_qsl(parse_result.query))\n query.setdefault(\"rate\", str(wav_file.getframerate()))\n query.setdefault(\"width\", str(wav_file.getsampwidth()))\n query.setdefault(\"channels\", str(wav_file.getnchannels()))\n\n url = urlunparse(parse_result._replace(query=urlencode(query)))\n async with connect(url) as websocket:\n chunk = wav_file.readframes(args.samples_per_chunk)\n while chunk:\n await websocket.send(chunk)\n chunk = wav_file.readframes(args.samples_per_chunk)\n\n # Signal stop with empty message\n await websocket.send(bytes())\n result = await websocket.recv()\n print(result)\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n" }, { "path": "tools/websocket-client/requirements.txt", "content": "websockets\n" }, { "path": "tools/websocket-client/script/run", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\nif [ -d \"${venv}\" ]; then\n source \"${venv}/bin/activate\"\nfi\n\nexport PATH=\"${base_dir}/bin:${PATH}\"\n\npython3 \"${base_dir}/bin/websocket_client.py\" \"$@\"\n" }, { "path": "tools/websocket-client/script/setup", "content": "#!/usr/bin/env bash\nset -eo pipefail\n\n# Directory of *this* script\nthis_dir=\"$( cd \"$( dirname \"$0\" )\" && pwd )\"\n\n# Base directory of repo\nbase_dir=\"$(realpath \"${this_dir}/..\")\"\n\n# Path to virtual environment\n: \"${venv:=${base_dir}/.venv}\"\n\n# Python binary to use\n: \"${PYTHON=python3}\"\n\npython_version=\"$(${PYTHON} --version)\"\n\n# Create virtual environment\necho \"Creating virtual environment at ${venv} (${python_version})\"\nrm -rf \"${venv}\"\n\"${PYTHON}\" -m venv \"${venv}\"\nsource \"${venv}/bin/activate\"\n\n# Install Python dependencies\necho 'Installing Python dependencies'\npip3 install --upgrade pip\npip3 install --upgrade wheel setuptools\n\npip3 install -r \"${base_dir}/requirements.txt\"\n\n# -----------------------------------------------------------------------------\n\necho \"OK\"\n" } ]