Repository: daanzu/kaldi-active-grammar Branch: master Commit: 2e4aafae406b Files: 54 Total size: 470.6 KB Directory structure: gitextract_2q213gax/ ├── .github/ │ ├── FUNDING.yml │ ├── RELEASING.md │ ├── release_notes.md │ └── workflows/ │ └── build.yml ├── .gitignore ├── AGENTS.md ├── CHANGELOG.md ├── CMakeLists.txt ├── Justfile ├── LICENSE.txt ├── README.md ├── building/ │ ├── build-wheel-dockcross.sh │ ├── dockcross-manylinux2010-x64 │ └── kaldi-configure-wrapper.sh ├── docs/ │ └── models.md ├── examples/ │ ├── audio.py │ ├── full_example.py │ ├── mix_dictation.py │ ├── plain_dictation.py │ ├── requirements_audio.txt │ └── util.py ├── kaldi_active_grammar/ │ ├── LICENSE.txt │ ├── __init__.py │ ├── __main__.py │ ├── compiler.py │ ├── defaults.py │ ├── ffi.py │ ├── kaldi/ │ │ ├── COPYING │ │ ├── __init__.py │ │ ├── augment_phones_txt.py │ │ ├── augment_phones_txt_py2.py │ │ ├── augment_words_txt.py │ │ ├── augment_words_txt_py2.py │ │ ├── make_lexicon_fst.py │ │ └── make_lexicon_fst_py2.py │ ├── model.py │ ├── plain_dictation.py │ ├── utils.py │ ├── wfst.py │ └── wrapper.py ├── pyproject.toml ├── requirements-build.txt ├── requirements-editable.txt ├── requirements-test.txt ├── setup.cfg ├── setup.py └── tests/ ├── conftest.py ├── generate_google_tts.py ├── generate_piper_tts.py ├── helpers.py ├── run_each_test_separately.py ├── test_grammar.py ├── test_package.py └── test_plain_dictation.py ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/FUNDING.yml ================================================ github: daanzu custom: "https://paypal.me/daanzu" ================================================ FILE: .github/RELEASING.md ================================================ # Release Process ## Quick Summary 1. **Prepare**: Update version in `__init__.py`, update `CHANGELOG.md`, run tests 2. **Tag Fork First**: Push all changes and tag `kag-vX.Y.Z` in kaldi-fork repo (MUST be done first!) 3. **Tag Main**: Create and push git tag `vX.Y.Z` in this repo to trigger builds 4. **Build**: Automated GitHub Actions builds wheels for all platforms 5. **Release**: Create GitHub release with changelog, upload wheel artifacts and models 6. **Publish**: Download wheels from artifacts, upload to PyPI with twine 7. **Finalize**: Bump to next dev version, announce release --- ## Detailed Release Process ### Overview This is a **duorepo** (2 separate repos used together): - Main repo: `daanzu/kaldi-active-grammar` (Python package) - Native binaries repo: `daanzu/kaldi-fork-active-grammar` (Kaldi C++ fork) **⚠️ IMPORTANT: The fork repo MUST always be pushed first before this repo for any changes!** The build process in this repo checks out code from the fork repo, so the fork must contain all necessary changes before triggering builds here. ### 1. Pre-Release Preparation #### Update Version Number Edit `kaldi_active_grammar/__init__.py`: ```python __version__ = 'X.Y.Z' # Change from previous version ``` Optionally update `REQUIRED_MODEL_VERSION` if model changes. #### Update CHANGELOG.md Add new version section following Keep a Changelog format: ```markdown ## [X.Y.Z](release-url) - YYYY-MM-DD - Changes: [KaldiAG](compare-url) [KaldiFork](compare-url) ### Added - New features ### Changed - Changes to existing functionality ### Fixed - Bug fixes ### Removed - Removed features ``` Include comparison links for both repos (KaldiAG and KaldiFork). #### Run Tests ```bash just test ``` #### Commit Changes ```bash git add kaldi_active_grammar/__init__.py CHANGELOG.md git commit -m "Release vX.Y.Z" ``` ### 2. Create Git Tags **⚠️ CRITICAL ORDER: Tag and push the fork repo FIRST, then this repo!** #### Tag the Kaldi Fork Repo (DO THIS FIRST!) In the `daanzu/kaldi-fork-active-grammar` repo: 1. Ensure all native code changes are committed and pushed 2. Create and push the tag: ```bash git tag kag-vX.Y.Z # Note the 'kag-' prefix matching the version git push origin kag-vX.Y.Z ``` This is crucial because the build process in this repo will check out code from the fork using this tag. #### Tag This Repo (DO THIS SECOND!) Only after the fork repo tag is pushed: ```bash git tag vX.Y.Z git push origin vX.Y.Z ``` Pushing this tag will trigger the GitHub Actions build workflow, which will pull the native code from the fork repo using the `kag-vX.Y.Z` tag. ### 3. Automated Build Process (GitHub Actions) When you push a tag, the CI automatically: #### Detects the Tag Sets `KALDI_BRANCH=kag-vX.Y.Z` for tagged commits, or uses current branch name for non-tagged commits. #### Builds Native Binaries **Linux** (`build-linux` job): - Uses dockcross/manylinux2010 for compatibility - Compiles Kaldi C++ code with CMake - Runs auditwheel for wheel repair **Windows** (`build-windows` job): - Uses Visual Studio 2022/2025 - Installs Intel MKL - Compiles OpenFST and Kaldi with MSBuild **macOS ARM** (`build-macos-arm` job): - For Apple Silicon (M1/M2/etc) - Uses delocate for wheel repair **macOS Intel** (`build-macos-intel` job): - For x86_64 Macs - Uses delocate for wheel repair #### Caches Native Binaries Caches compiled Kaldi binaries by commit hash to speed up rebuilds. #### Creates Python Wheels - Packages include all native binaries - Platform-specific wheels: `py3-none-{platform}` - Uses `setup.py` with scikit-build (or `KALDIAG_BUILD_SKIP_NATIVE=1` for packaging only) #### Tests All Wheels `test-wheels` job runs on multiple OS/Python version combinations. #### Merges Wheel Artifacts `merge-wheels` job combines all platform wheels into single artifact named `wheels`. ### 4. Manual Workflow Trigger (Optional) You can also trigger builds manually: ```bash # Using GitHub CLI gh workflow run build.yml --ref master # Or specific ref: gh workflow run build.yml --ref vX.Y.Z # Or via Justfile: just trigger-build master ``` ### 5. Create GitHub Release 1. **Navigate to GitHub Releases page** - https://github.com/daanzu/kaldi-active-grammar/releases 2. **Create new release**: - Tag: `vX.Y.Z` (select existing tag) - Title: `vX.Y.Z` or descriptive name - Description: Copy relevant section from `CHANGELOG.md` or use template from `.github/release_notes.md` 3. **Download wheel artifacts** from successful build workflow: - Go to Actions → Build workflow → successful run - Download artifact named `wheels` (merged) or individual `wheels-{platform}` 4. **Upload wheels to release**: - Upload all `.whl` files from the artifacts 5. **Upload additional assets** (if applicable): - Pre-trained Kaldi models (if updated) - WinPython distributions (if prepared): - `kaldi-dragonfly-winpython` (stable) - `kaldi-dragonfly-winpython-dev` (development) - `kaldi-caster-winpython-dev` (with Caster) 6. **Publish the release** ### 6. Publish to PyPI The process is currently **manual** (not automated in workflow). #### Download Wheel Artifacts Download the `wheels` artifact from the successful GitHub Actions build. #### Upload to PyPI You'll need PyPI credentials (entered interactively or configured in `~/.pypirc` or via environment variables). ```bash # Test PyPI first (recommended): uvx twine upload --repository testpypi wheels/* # Production PyPI: uvx twine upload wheels/* ``` Or: ```bash pip install twine # Test PyPI first (recommended): twine upload --repository testpypi wheels/* # Production PyPI: twine upload wheels/* ``` #### Verify on PyPI - Check https://pypi.org/project/kaldi-active-grammar/ - Verify all platforms are present - Test installation: ```bash pip install kaldi-active-grammar==X.Y.Z ``` ### 7. Post-Release Tasks #### Bump Version for Development Update `__version__` in `kaldi_active_grammar/__init__.py` to next dev version: ```python __version__ = 'X.Y.Z.dev0' # or 'X.Y+1.0.dev0' ``` Commit the change: ```bash git add kaldi_active_grammar/__init__.py git commit -m "Bump version to X.Y.Z.dev0" git push ``` #### Announce Release - Update documentation/README if needed - Update Dragonfly documentation if relevant - Post on relevant forums/communities - Notify on Gitter: - https://app.gitter.im/#/room/#dragonfly2:matrix.org - https://gitter.im/kaldi-active-grammar/community --- ## Key Files in Release Process | File | Purpose | |------|---------| | `kaldi_active_grammar/__init__.py:8` | Version source | | `kaldi_active_grammar/__init__.py:10` | Required model version | | `CHANGELOG.md` | Release notes and history | | `.github/workflows/build.yml` | CI build configuration | | `.github/workflows/tests.yml` | CI test configuration | | `setup.py` | Package build configuration | | `pyproject.toml` | Build system requirements | | `Justfile` | Build and test tasks | | `.github/release_notes.md` | Release notes template | --- ## Environment Variables | Variable | Purpose | |----------|---------| | `KALDIAG_BUILD_SKIP_NATIVE=1` | Skip native compilation, just package | | `KALDI_BRANCH` | Which Kaldi fork branch/tag to build from (auto-detected from git tag) | | `MKL_URL` | Optional Intel MKL download URL (mostly disabled now) | --- ## Development vs Release Versions - **Dev versions**: `setup.py` auto-appends timestamp to `X.Y.Z.dev0` versions - Example: `3.1.0.dev20251031123456` - **Release versions**: Clean `X.Y.Z` semantic version - Example: `3.1.0` - Build process differentiates based on git tags --- ## Troubleshooting ### Build fails on one platform - Check the GitHub Actions logs for that specific job - Native binaries are cached, so may need to invalidate cache if Kaldi fork changed - Ensure the `kag-vX.Y.Z` tag exists in kaldi-fork-active-grammar repo ### Tests fail - Run tests locally: `just test` - Check if model needs to be updated - Verify test data is downloaded: `just setup-tests` ### PyPI upload fails - Verify credentials in `~/.pypirc` - Check wheel filenames are correct - Ensure version doesn't already exist on PyPI - Try test PyPI first ### Wheels missing for a platform - Check if that build job completed successfully - Look for cache issues - Verify platform is included in build matrix ### Version mismatch - Ensure git tag matches `__version__` in `__init__.py` - Check that both repos are tagged (main repo: `vX.Y.Z`, fork: `kag-vX.Y.Z`) - Verify `CHANGELOG.md` has correct version ================================================ FILE: .github/release_notes.md ================================================ v0.5.0: User Lexicon! Compilation Optimizations! Better Model! ### Notes * **User Lexicon**: you can add new words/pronunciations to the model's lexicon to be recognized & used in grammars, and the pronunciations can be either specified explicitly or inferred automatically. * **Compilation Optimizations**: compilation while loading grammars uses the disk much less, and far fewer passes are made over the graphs, as separate modules have been customized & combined. * **Better Model**: 50% more training data. ### Artifacts * **`kaldi_model_zamia`**: [*new model version required!*] A compatible general English Kaldi nnet3 chain model. * **`kaldi-dragonfly-winpython`**: A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-dragonfly-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! ### **Donations are appreciated to encourage development.** [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) v0.6.0: Big Fixes And Optimizations To Get Caster Running ### Artifacts * **`kaldi_model_zamia`**: A compatible general English Kaldi nnet3 chain model. * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-dragonfly-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### **Donations are appreciated to encourage development.** [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) v0.7.1: Partial Decoding, Parallel Compilation, & Various Optimizations for 15-50% Speedup Support is now included in dragonfly2 v0.17.0! You can try a self-contained distribution available below, of either stable or development versions. ### Notes * **Partial Decoding**: support for having **separate Voice Activity Detection timeout values** based on whether the current utterance is complex (dictation) or not. * **Parallel Compilation**: when compiling grammars/rules that are not cached, multiple can be compiled at once (up to your core count). * Example: loading Caster without cache is ~40% faster (in addition to optimizations below). * **Various Optimizations**: loading even while cached sped up 15%. * Refactored temporary/cache file handling * Various bug fixes ### Artifacts * **`kaldi_model_zamia`**: A compatible general English Kaldi nnet3 chain model. * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-dragonfly-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### **Donations are appreciated to encourage development.** [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) v1.0.0: Faster Loading, Python3, Grammar/Rule Weights, and more Support is now included in dragonfly2 v0.18.0! You can try a self-contained distribution available below, of either stable or development versions. ### Notes * **Direct Parsing**: parse recognitions directly on the FST, removing the (slow) `pyparsing` dependency. * Caster example: Loading is now **~50%** faster when cached, and the Kaldi backend accounts for only ~15% of loading time. * **Python3**: both python 2 and 3 should be fully supported now. * **Unicode**: this should also fix unicode issues in various places in both python2/3. * **Grammar/Rule Weights**: can specify weight, where grammars/rules with higher weight value are more likely to be recognized, compared to their peers, for an ambiguous recognition. * **Generalized Alternative Dictation**: the cloud dictation feature has been generalized to make it easier to add other alternatives in the future. * Various bug fixes & optimizations ### Artifacts * **`kaldi_model_zamia`**: A compatible general English Kaldi nnet3 chain model. * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-dragonfly-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-pink.svg)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/preferred-GitHub-black.svg)](https://github.com/sponsors/daanzu) [**GitHub** is currently matching all my donations $-for-$.] v1.2.0: Improved Recognition, Weights on Any Elements, Pluggable Alternative Dictation, Stand-alone Plain Dictation Interface, and More Support is now included in dragonfly2 v0.20.0! You can try a self-contained distribution available below, of either stable or development versions. ### Notes * **Improved Recognition**: better graph construction/compilation should give significantly better overall recognition. * **Weights on Any Elements**: you can now easily add weights to any element (including compound elements in `MappingRule`s), in addition to any rule/grammar. * **Pluggable Alternative Dictation**: you can optionally pass a `callable` as `alternative_dictation` to define your own, external dictation engine. * **Stand-alone Plain Dictation Interface**: the library now provides a simple interface for recognizing plain dictation without fancy active grammar features. * **NOTE**: the default model directory is now `kaldi_model`. * Various bug fixes & optimizations ### Artifacts * **`kaldi_model_daanzu`**: A better overall compatible general English Kaldi nnet3 chain model than below. * **`kaldi_model_zamia_daanzu_mediumlm`**: A compatible general English Kaldi nnet3 chain model, with a larger/better dictation language model than below. * **`kaldi_model_zamia`**: A compatible general English Kaldi nnet3 chain model. * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-dragonfly-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-pink.svg)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/preferred-GitHub-black.svg)](https://github.com/sponsors/daanzu) [**GitHub** is currently matching all my donations $-for-$.] v1.3.0: Preparation and Fixes for Next Generation of Models This should be included the next dragonfly version, or you can try a self-contained distribution available below. You can subscribe to announcements on Gitter: see [instructions](https://gitlab.com/gitlab-org/gitter/webapp/blob/master/docs/notifications.md#announcements). [![Gitter](https://badges.gitter.im/kaldi-active-grammar/community.svg)](https://gitter.im/kaldi-active-grammar/community) ### Notes * **Next Generation of Models**: support for a new generation of models, trained on more data, and with hopefully better accuracy. * **User Lexicon**: if there is a ``user_lexicon.txt`` file in the current working directory of your initial loader script, its contents will be automatically added to the ``user_lexicon.txt`` in the active model when it is loaded. * Various bug fixes & optimizations ### Artifacts * **`kaldi_model_daanzu*`**: A better acoustic model, and varying levels of language model for dictation (bigger is generally better). * **`kaldi_model_zamia`**: A compatible general English Kaldi nnet3 chain model. * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-dragonfly-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-pink.svg)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/preferred-GitHub-black.svg)](https://github.com/sponsors/daanzu) [**GitHub** is matching (only) my **GitHub Sponsors** donations.] v1.4.0: MacOS Support, And Faster Graph Compilation Support is now included in dragonfly2 v0.22.0! You can try a self-contained distribution available below. You can subscribe to announcements on Gitter: see [instructions](https://gitlab.com/gitlab-org/gitter/webapp/blob/master/docs/notifications.md#announcements). [![Gitter](https://badges.gitter.im/kaldi-active-grammar/community.svg)](https://gitter.im/kaldi-active-grammar/community) ### Notes * **MacOS Support** * **Faster Graph Compilation** * **Dictation**: the dictation model now does not recognize a zero-word sequence * Various bug fixes & optimizations ### Artifacts * **`kaldi_model_daanzu*`**: A better acoustic model, and varying levels of language model for dictation (bigger is generally better). * **`kaldi_model_zamia`**: A compatible general English Kaldi nnet3 chain model. * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-pink.svg)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/preferred-GitHub-black.svg)](https://github.com/sponsors/daanzu) [**GitHub** is matching (only) my **GitHub Sponsors** donations.] v1.5.0: Improved Recognition Confidence Estimation You can subscribe to announcements on Gitter: see [instructions](https://gitlab.com/gitlab-org/gitter/webapp/blob/master/docs/notifications.md#announcements). [![Gitter](https://badges.gitter.im/kaldi-active-grammar/community.svg)](https://gitter.im/kaldi-active-grammar/community) ### Notes * **Improved Recognition Confidence Estimation**: two new, different measures: * `confidence`: basically the difference in how much "better" the returned recognition was, compared to the second best guess (`>0`) * `expected_error_rate`: an estimate of how often similar utterances are incorrect (roughly out of `1.0`, but can be greater) * Refactoring in preparation for future improvements * Various bug fixes & optimizations ### Artifacts * **Models are available [here](https://github.com/daanzu/kaldi-active-grammar/blob/master/docs/models.md)** * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-pink.svg)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/preferred-GitHub-black.svg)](https://github.com/sponsors/daanzu) [**GitHub** is matching (only) my **GitHub Sponsors** donations.] v1.6.0: Easier Configuration; Public Automated Builds You can subscribe to announcements on GitHub (see Watch panel above), or on Gitter (see [instructions](https://gitlab.com/gitlab-org/gitter/webapp/blob/master/docs/notifications.md#announcements) [![Gitter](https://badges.gitter.im/kaldi-active-grammar/community.svg)](https://gitter.im/kaldi-active-grammar/community)) ### Added * Can now pass configuration dict to `KaldiAgfNNet3Decoder`, `PlainDictationRecognizer` (without `HCLG.fst`). * Continuous Integration builds run on GitHub Actions for Windows (x64), MacOS (x64), Linux (x64). ### Changed * Refactor of passing configuration to initialization. * `PlainDictationRecognizer.decode_utterance` can take `chunk_size` parameter. * Smaller binaries: MacOS 11MB -> 7.6MB, Linux 21MB -> 18MB. ### Fixed * Confidence measurement in the presence of multiple, redundant rules. * Python3 int division bug for cloud dictation. ### Artifacts * **Models are available [here](https://github.com/daanzu/kaldi-active-grammar/blob/master/docs/models.md)** * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. ### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-pink.svg)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/preferred-GitHub-black.svg)](https://github.com/sponsors/daanzu) [**GitHub** is matching (only) my **GitHub Sponsors** donations.] v2.0.0: Faster Grammar Compilation; Cleaner Codebase; Preparation For New Features v2.1.0 Minor fix for OpenBLAS compilation for some architectures on linux/mac. See [major changes introduced in v2.0.0 and associated downloads](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v2.0.0). v3.2.0 Functionality-wise, only one small bug-fix: to the broken alternative dictation interface. But extensive build and infrastructure changes lead me to make this a minor release rather than just a patch release, out of an abundance of caution. Active development has resumed after a long break! (While development paused, the project was continuously maintained and actively used in production.) Look forward to more frequent releases in the hopefully-near future. You can subscribe to announcements on GitHub (see Watch panel above), or on Gitter (see [instructions](https://gitlab.com/gitlab-org/gitter/webapp/blob/master/docs/notifications.md#announcements) [![Gitter](https://badges.gitter.im/kaldi-active-grammar/community.svg)](https://gitter.im/kaldi-active-grammar/community)) #### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-002991.svg?logo=paypal)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) ### Artifacts * **Models are available [here](https://github.com/daanzu/kaldi-active-grammar/blob/master/docs/models.md)** and below. * **`kaldi-dragonfly-winpython`**: A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython`**: A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! If you have trouble downloading, try using `wget --continue`. * **`kaldi-dragonfly-winpython`**: [*stable release version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! This should be included the next dragonfly version, or you can try a self-contained distribution available below. ================================================ FILE: .github/workflows/build.yml ================================================ name: Build on: push: pull_request: workflow_dispatch: # gh api repos/:owner/:repo/actions/workflows/build.yml/dispatches -F ref=master # gh workflow run build.yml --ref develop # gh workflow run build.yml jobs: build-linux: runs-on: ubuntu-latest if: true env: MKL_URL: "" # MKL_URL: "https://registrationcenter-download.intel.com/akdlm/irc_nas/tec/16917/l_mkl_2020.4.304.tgz" steps: - name: Checkout repository uses: actions/checkout@v5 - name: Get KALDI_BRANCH (kag-$TAG tag if commit is tagged; current branch name if not) run: | # Fetch tags on the one fetched commit (shallow clone) git fetch --depth=1 origin "+refs/tags/*:refs/tags/*" export TAG=$(git tag --points-at HEAD) echo "TAG: $TAG" if [[ $TAG ]]; then echo "KALDI_BRANCH: kag-$TAG" echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_ENV echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_OUTPUT else echo "KALDI_BRANCH: ${GITHUB_REF/refs\/heads\//}" echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_ENV echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_OUTPUT fi - name: Get Kaldi commit hash id: get-kaldi-commit run: | KALDI_COMMIT=$(git ls-remote https://github.com/daanzu/kaldi-fork-active-grammar.git $KALDI_BRANCH | cut -f1) echo "KALDI_COMMIT: $KALDI_COMMIT" echo "KALDI_COMMIT=$KALDI_COMMIT" >> $GITHUB_OUTPUT - name: Restore cached native binaries id: cache-native-binaries-restore uses: actions/cache/restore@v4 with: key: native-${{ runner.os }}-${{ steps.get-kaldi-commit.outputs.KALDI_COMMIT }}-${{ env.MKL_URL }}-v1 path: | kaldi_active_grammar/exec/linux kaldi_active_grammar.libs - name: Install just uses: taiki-e/install-action@just - name: Build with dockcross (native binaries & python wheel) run: | shopt -s nullglob echo "KALDI_BRANCH: $KALDI_BRANCH" echo "MKL_URL: $MKL_URL" just build-dockcross ${{ steps.cache-native-binaries-restore.outputs.cache-hit == 'true' && '--skip-native' || '' }} $KALDI_BRANCH $MKL_URL ls -l wheelhouse/ for whl in wheelhouse/*.whl; do unzip -l $whl done - name: Extract native binaries from wheel after auditwheel repair, to save to cache if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' # We do this rather than manually caching all of the various kaldi/openfst libraries in their build locations run: | shopt -s nullglob # Assert there is only one wheel WHEEL_COUNT=$(ls wheelhouse/*.whl | wc -l) if [ "$WHEEL_COUNT" -ne 1 ]; then echo "Error: Expected exactly 1 wheel, found $WHEEL_COUNT" ls -l wheelhouse/ exit 1 fi WHEEL_FILE=$(ls wheelhouse/*.whl) echo "Extracting from wheel: $WHEEL_FILE" unzip -o $WHEEL_FILE 'kaldi_active_grammar/exec/linux/*' unzip -o $WHEEL_FILE 'kaldi_active_grammar.libs/*' ls -l kaldi_active_grammar/exec/linux/ kaldi_active_grammar.libs/ readelf -d kaldi_active_grammar/exec/linux/libkaldi-dragonfly.so | egrep 'NEEDED|RUNPATH|RPATH' - name: Save cached native binaries if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' uses: actions/cache/save@v4 with: key: ${{ steps.cache-native-binaries-restore.outputs.cache-primary-key }} path: | kaldi_active_grammar/exec/linux kaldi_active_grammar.libs - name: Upload native binaries to artifacts uses: actions/upload-artifact@v4 with: name: native-linux path: | kaldi_active_grammar/exec/linux kaldi_active_grammar.libs - name: Upload Linux wheels uses: actions/upload-artifact@v4 with: name: wheels-linux path: wheelhouse/* - name: Examine results run: | shopt -s nullglob for whl in wheelhouse/*.whl; do echo "::notice title=Built wheel::$(basename $whl)" unzip -l $whl done build-windows: runs-on: windows-2025 if: true env: VS_VERSION: vs2022 PLATFORM_TOOLSET: v143 WINDOWS_TARGET_PLATFORM_VERSION: 10.0 MKL_VERSION: 2025.1.0 defaults: run: shell: bash steps: - name: Checkout main repository uses: actions/checkout@v5 with: path: main - name: Get KALDI_BRANCH (kag-$TAG tag if commit is tagged; current branch name if not) id: get-kaldi-branch working-directory: main run: | # Fetch tags on the one fetched commit (shallow clone) git fetch --depth=1 origin "+refs/tags/*:refs/tags/*" export TAG=$(git tag --points-at HEAD) echo "TAG: $TAG" if [[ $TAG ]]; then echo "KALDI_BRANCH: kag-$TAG" echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_ENV echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_OUTPUT else echo "KALDI_BRANCH: ${GITHUB_REF/refs\/heads\//}" echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_ENV echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_OUTPUT fi - name: Get Kaldi commit hash id: get-kaldi-commit run: | KALDI_COMMIT=$(git ls-remote https://github.com/daanzu/kaldi-fork-active-grammar.git $KALDI_BRANCH | cut -f1) echo "KALDI_COMMIT: $KALDI_COMMIT" echo "KALDI_COMMIT=$KALDI_COMMIT" >> $GITHUB_OUTPUT - name: Restore cached native binaries id: cache-native-binaries-restore uses: actions/cache/restore@v4 with: key: native-${{ runner.os }}-${{ steps.get-kaldi-commit.outputs.KALDI_COMMIT }}-${{ env.VS_VERSION }}-${{ env.PLATFORM_TOOLSET }}-${{ env.WINDOWS_TARGET_PLATFORM_VERSION }}-${{ env.MKL_VERSION }}-v1 path: main/kaldi_active_grammar/exec/windows - name: Checkout OpenFST repository if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' uses: actions/checkout@v5 with: repository: daanzu/openfst path: openfst - name: Checkout Kaldi repository if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' uses: actions/checkout@v5 with: repository: daanzu/kaldi-fork-active-grammar path: kaldi ref: ${{ steps.get-kaldi-branch.outputs.KALDI_BRANCH }} - name: Gather system information if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' run: | echo $GITHUB_WORKSPACE df -h echo "Windows SDK Versions:" ls -l '/c/Program Files (x86)/Windows Kits/10/Include/' echo "Visual Studio Redistributables:" # ls -l '/c/Program Files (x86)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/' # ls -l '/c/Program Files (x86)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/14.26.28720' # ls -l '/c/Program Files (x86)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/v142' # ls -l '/c/Program Files (x86)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/14.16.27012/x64/Microsoft.VC141.CRT' # ls -l '/c/Program Files (x86)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/'*/x64/Microsoft.*.CRT # ls -lR /c/Program\ Files\ \(x86\)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/ # ls -lR '/c/Program Files (x86)/Microsoft Visual Studio/'2022/Enterprise/VC/Redist/MSVC/ vswhere vswhere -find 'VC\Redist\**\VC_redist.x64.exe' - name: Setup Kaldi build configuration if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' run: | cd kaldi/windows cp kaldiwin_mkl.props kaldiwin.props cp variables.props.dev variables.props # Set openfst location perl -pi -e 's/.*<\/OPENFST>/$ENV{GITHUB_WORKSPACE}\\openfst<\/OPENFST>/g' variables.props perl -pi -e 's/.*<\/OPENFSTLIB>/$ENV{GITHUB_WORKSPACE}\\openfst\\build_output<\/OPENFSTLIB>/g' variables.props perl generate_solution.pl --vsver ${VS_VERSION} --enable-mkl --noportaudio # Add additional libfstscript library to dragonfly build file sed -i.bak '$i\ \ \ libfstscript.lib;%(AdditionalDependencies)\ \ ' ../kaldiwin_${VS_VERSION}_MKL/kaldiwin/kaldi-dragonfly/kaldi-dragonfly.vcxproj perl get_version.pl - name: Add msbuild to PATH if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' uses: microsoft/setup-msbuild@v2 - name: Install MKL if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' run: winget install --id=Intel.oneMKL -v "${MKL_VERSION}" -e --accept-package-agreements --accept-source-agreements --disable-interactivity - name: Build OpenFST if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' shell: cmd run: msbuild -t:Build -p:Configuration=Release -p:Platform=x64 -p:PlatformToolset=%PLATFORM_TOOLSET% -maxCpuCount -verbosity:minimal openfst/openfst.sln - name: Build Kaldi if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' shell: cmd run: msbuild -t:Build -p:Configuration=Release -p:Platform=x64 -p:PlatformToolset=%PLATFORM_TOOLSET% -p:WindowsTargetPlatformVersion=%WINDOWS_TARGET_PLATFORM_VERSION% -maxCpuCount -verbosity:minimal kaldi/kaldiwin_%VS_VERSION%_MKL/kaldiwin/kaldi-dragonfly/kaldi-dragonfly.vcxproj - name: Copy native binaries if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' run: | mkdir -p main/kaldi_active_grammar/exec/windows cp kaldi/kaldiwin_${VS_VERSION}_MKL/kaldiwin/kaldi-dragonfly/x64/Release/kaldi-dragonfly.dll main/kaldi_active_grammar/exec/windows/ - name: Save cached native binaries if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' uses: actions/cache/save@v4 with: key: ${{ steps.cache-native-binaries-restore.outputs.cache-primary-key }} path: main/kaldi_active_grammar/exec/windows - name: Upload native binaries to artifacts uses: actions/upload-artifact@v4 with: name: native-windows path: main/kaldi_active_grammar/exec/windows - name: Build Python wheel working-directory: main run: | python -m pip -V python -m pip install --upgrade setuptools wheel env KALDIAG_BUILD_SKIP_NATIVE=1 python setup.py bdist_wheel ls -l dist/ - name: Upload Windows wheels uses: actions/upload-artifact@v4 with: name: wheels-windows path: main/dist/* - name: Examine results run: | shopt -s nullglob for whl in main/dist/*.whl; do echo "::notice title=Built wheel::$(basename $whl)" unzip -l $whl done # - name: Copy Windows vc_redist # run: | # mkdir -p vc_redist # cp '/c/Program Files (x86)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/14.26.28720'/vc_redist.x64.exe vc_redist/ # cp '/c/Program Files (x86)/Microsoft Visual Studio/2022/Enterprise/VC/Redist/MSVC/14.26.28720'/x64/Microsoft.*.CRT/* vc_redist/ # - uses: actions/upload-artifact@v4 # with: # name: windows_vc_redist # path: vc_redist/* build-macos-arm: runs-on: macos-15 if: true env: MACOSX_DEPLOYMENT_TARGET: "11.0" MKL_URL: "" # MKL_URL: https://registrationcenter-download.intel.com/akdlm/irc_nas/tec/17172/m_mkl_2020.4.301.dmg steps: - name: Checkout repository uses: actions/checkout@v5 - name: Get KALDI_BRANCH (kag-$TAG tag if commit is tagged; current branch name if not) id: get-kaldi-branch run: | # Fetch tags on the one fetched commit (shallow clone) git fetch --depth=1 origin "+refs/tags/*:refs/tags/*" export TAG=$(git tag --points-at HEAD) echo "TAG: $TAG" if [[ $TAG ]]; then echo "KALDI_BRANCH: kag-$TAG" echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_ENV echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_OUTPUT else echo "KALDI_BRANCH: ${GITHUB_REF/refs\/heads\//}" echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_ENV echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_OUTPUT fi - name: Get Kaldi commit hash id: get-kaldi-commit run: | KALDI_COMMIT=$(git ls-remote https://github.com/daanzu/kaldi-fork-active-grammar.git $KALDI_BRANCH | cut -f1) echo "KALDI_COMMIT: $KALDI_COMMIT" echo "KALDI_COMMIT=$KALDI_COMMIT" >> $GITHUB_OUTPUT - name: Restore cached native binaries id: cache-native-binaries-restore uses: actions/cache/restore@v4 with: key: native-${{ runner.os }}-arm-${{ steps.get-kaldi-commit.outputs.KALDI_COMMIT }}-${{ env.MACOSX_DEPLOYMENT_TARGET }}-${{ env.MKL_URL }}-v1 path: kaldi_active_grammar/exec/macos - name: Install MKL (if enabled) if: ${{ env.MKL_URL != '' && steps.cache-native-binaries-restore.outputs.cache-hit != 'true' }} run: | echo "Installing MKL from: $MKL_URL" export MKL_FILE=${MKL_URL##*/} export MKL_FILE=${MKL_FILE%\.dmg} wget --no-verbose $MKL_URL hdiutil attach ${MKL_FILE}.dmg cp /Volumes/${MKL_FILE}/${MKL_FILE}.app/Contents/MacOS/silent.cfg . sed -i.bak -e 's/decline/accept/g' silent.cfg sudo /Volumes/${MKL_FILE}/${MKL_FILE}.app/Contents/MacOS/install.sh --silent silent.cfg - name: Install dependencies for native build if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' run: | python3 -m pip install --break-system-packages --user --upgrade scikit-build>=0.10.0 cmake ninja brew install automake sox libtool brew reinstall gfortran # For openblas # brew install autoconf - name: Install dependencies for python build run: | python3 -m pip install --break-system-packages --user --upgrade setuptools wheel delocate - name: Build Python wheel run: | shopt -s nullglob echo "KALDI_BRANCH: $KALDI_BRANCH" echo "MKL_URL: $MKL_URL" ${{ steps.cache-native-binaries-restore.outputs.cache-hit == 'true' && 'KALDIAG_BUILD_SKIP_NATIVE=1' || '' }} python3 setup.py bdist_wheel ls -l dist/ for whl in dist/*.whl; do unzip -l $whl done - name: Repair wheel with delocate run: | shopt -s nullglob for whl in dist/*.whl; do echo "Examining wheel before delocate: $whl" python3 -m delocate.cmd.delocate_listdeps -d $whl echo "Repairing wheel: $whl" python3 -m delocate.cmd.delocate_wheel -v -w wheelhouse -L exec/macos/libs --require-archs arm64 $whl done # NOTE: This also downgrades the required MacOS version to the minimum possible ls -l wheelhouse/ for whl in wheelhouse/*.whl; do echo "Examining repaired wheel: $whl" python3 -m delocate.cmd.delocate_listdeps -d $whl done - name: Extract native binaries from wheel after delocate repair, to save to cache if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' # We do this rather than manually caching all of the various kaldi/openfst libraries in their build locations run: | shopt -s nullglob # Assert there is only one wheel WHEEL_COUNT=$(ls wheelhouse/*.whl | wc -l) if [ "$WHEEL_COUNT" -ne 1 ]; then echo "Error: Expected exactly 1 wheel, found $WHEEL_COUNT" ls -l wheelhouse/ exit 1 fi WHEEL_FILE=$(ls wheelhouse/*.whl) echo "Extracting from wheel: $WHEEL_FILE" unzip -o $WHEEL_FILE 'kaldi_active_grammar/exec/macos/*' ls -l kaldi_active_grammar/exec/macos/ otool -l kaldi_active_grammar/exec/macos/libkaldi-dragonfly.dylib | egrep -A2 'LC_RPATH|cmd LC_LOAD_DYLIB' otool -L kaldi_active_grammar/exec/macos/libkaldi-dragonfly.dylib lipo -archs kaldi_active_grammar/exec/macos/libkaldi-dragonfly.dylib - name: Save cached native binaries if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' uses: actions/cache/save@v4 with: key: ${{ steps.cache-native-binaries-restore.outputs.cache-primary-key }} path: kaldi_active_grammar/exec/macos - name: Upload native binaries to artifacts uses: actions/upload-artifact@v4 with: name: native-macos-arm path: kaldi_active_grammar/exec/macos - name: Upload MacOS ARM wheels uses: actions/upload-artifact@v4 with: name: wheels-macos-arm path: wheelhouse/* - name: Examine results run: | shopt -s nullglob for whl in wheelhouse/*.whl; do echo "::notice title=Built wheel::$(basename $whl)" unzip -l $whl done build-macos-intel: runs-on: macos-15-intel if: true env: MACOSX_DEPLOYMENT_TARGET: "10.9" MKL_URL: "" # MKL_URL: https://registrationcenter-download.intel.com/akdlm/irc_nas/tec/17172/m_mkl_2020.4.301.dmg steps: - name: Checkout repository uses: actions/checkout@v5 - name: Get KALDI_BRANCH (kag-$TAG tag if commit is tagged; current branch name if not) id: get-kaldi-branch run: | # Fetch tags on the one fetched commit (shallow clone) git fetch --depth=1 origin "+refs/tags/*:refs/tags/*" export TAG=$(git tag --points-at HEAD) echo "TAG: $TAG" if [[ $TAG ]]; then echo "KALDI_BRANCH: kag-$TAG" echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_ENV echo "KALDI_BRANCH=kag-$TAG" >> $GITHUB_OUTPUT else echo "KALDI_BRANCH: ${GITHUB_REF/refs\/heads\//}" echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_ENV echo "KALDI_BRANCH=${GITHUB_REF/refs\/heads\//}" >> $GITHUB_OUTPUT fi - name: Get Kaldi commit hash id: get-kaldi-commit run: | KALDI_COMMIT=$(git ls-remote https://github.com/daanzu/kaldi-fork-active-grammar.git $KALDI_BRANCH | cut -f1) echo "KALDI_COMMIT: $KALDI_COMMIT" echo "KALDI_COMMIT=$KALDI_COMMIT" >> $GITHUB_OUTPUT - name: Restore cached native binaries id: cache-native-binaries-restore uses: actions/cache/restore@v4 with: key: native-${{ runner.os }}-intel-${{ steps.get-kaldi-commit.outputs.KALDI_COMMIT }}-${{ env.MACOSX_DEPLOYMENT_TARGET }}-${{ env.MKL_URL }}-v1 path: kaldi_active_grammar/exec/macos - name: Install MKL (if enabled) if: ${{ env.MKL_URL != '' && steps.cache-native-binaries-restore.outputs.cache-hit != 'true' }} run: | echo "Installing MKL from: $MKL_URL" export MKL_FILE=${MKL_URL##*/} export MKL_FILE=${MKL_FILE%\.dmg} wget --no-verbose $MKL_URL hdiutil attach ${MKL_FILE}.dmg cp /Volumes/${MKL_FILE}/${MKL_FILE}.app/Contents/MacOS/silent.cfg . sed -i.bak -e 's/decline/accept/g' silent.cfg sudo /Volumes/${MKL_FILE}/${MKL_FILE}.app/Contents/MacOS/install.sh --silent silent.cfg - name: Install dependencies for native build if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' run: | python3 -m pip install --break-system-packages --user --upgrade scikit-build>=0.10.0 cmake ninja brew install automake sox brew reinstall gfortran # For openblas # brew install autoconf libtool - name: Install dependencies for python build run: | python3 -m pip install --break-system-packages --user --upgrade setuptools wheel delocate - name: Build Python wheel run: | shopt -s nullglob echo "KALDI_BRANCH: $KALDI_BRANCH" echo "MKL_URL: $MKL_URL" ${{ steps.cache-native-binaries-restore.outputs.cache-hit == 'true' && 'KALDIAG_BUILD_SKIP_NATIVE=1' || '' }} python3 setup.py bdist_wheel ls -l dist/ for whl in dist/*.whl; do unzip -l $whl done - name: Repair wheel with delocate run: | shopt -s nullglob for whl in dist/*.whl; do echo "Examining wheel before delocate: $whl" python3 -m delocate.cmd.delocate_listdeps -d $whl echo "Repairing wheel: $whl" python3 -m delocate.cmd.delocate_wheel -v -w wheelhouse -L exec/macos/libs --require-archs x86_64 $whl done # NOTE: This also downgrades the required MacOS version to the minimum possible ls -l wheelhouse/ for whl in wheelhouse/*.whl; do echo "Examining repaired wheel: $whl" python3 -m delocate.cmd.delocate_listdeps -d $whl done - name: Extract native binaries from wheel after delocate repair, to save to cache if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' # We do this rather than manually caching all of the various kaldi/openfst libraries in their build locations run: | shopt -s nullglob # Assert there is only one wheel WHEEL_COUNT=$(ls wheelhouse/*.whl | wc -l) if [ "$WHEEL_COUNT" -ne 1 ]; then echo "Error: Expected exactly 1 wheel, found $WHEEL_COUNT" ls -l wheelhouse/ exit 1 fi WHEEL_FILE=$(ls wheelhouse/*.whl) echo "Extracting from wheel: $WHEEL_FILE" unzip -o $WHEEL_FILE 'kaldi_active_grammar/exec/macos/*' ls -l kaldi_active_grammar/exec/macos/ otool -l kaldi_active_grammar/exec/macos/libkaldi-dragonfly.dylib | egrep -A2 'LC_RPATH|cmd LC_LOAD_DYLIB' otool -L kaldi_active_grammar/exec/macos/libkaldi-dragonfly.dylib lipo -archs kaldi_active_grammar/exec/macos/libkaldi-dragonfly.dylib - name: Save cached native binaries if: steps.cache-native-binaries-restore.outputs.cache-hit != 'true' uses: actions/cache/save@v4 with: key: ${{ steps.cache-native-binaries-restore.outputs.cache-primary-key }} path: kaldi_active_grammar/exec/macos - name: Upload native binaries to artifacts uses: actions/upload-artifact@v4 with: name: native-macos-intel path: kaldi_active_grammar/exec/macos - name: Upload MacOS Intel wheels uses: actions/upload-artifact@v4 with: name: wheels-macos-intel path: wheelhouse/* - name: Examine results run: | shopt -s nullglob for whl in wheelhouse/*.whl; do echo "::notice title=Built wheel::$(basename $whl)" unzip -l $whl done merge-wheels: runs-on: ubuntu-latest needs: [ build-linux, build-windows, build-macos-arm, build-macos-intel, ] steps: - name: Merge all wheel artifacts uses: actions/upload-artifact/merge@v4 with: name: wheels pattern: wheels-* delete-merged: false # Optional: removes the individual artifacts setup-tests-cache: runs-on: ${{ matrix.os }} strategy: matrix: os: [ubuntu-latest, windows-latest] steps: - uses: actions/checkout@v5 - name: Install just uses: taiki-e/install-action@just - name: Install uv uses: astral-sh/setup-uv@v6 - name: Restore cached tests setup id: cache-tests-setup-restore uses: actions/cache/restore@v4 with: key: tests-setup-${{ hashFiles('Justfile') }}-v1 path: | tests/*.onnx tests/*.onnx.json tests/kaldi_model - name: Setup tests if: steps.cache-tests-setup-restore.outputs.cache-hit != 'true' run: | just setup-tests - name: Save cached tests setup if: steps.cache-tests-setup-restore.outputs.cache-hit != 'true' uses: actions/cache/save@v4 with: key: ${{ steps.cache-tests-setup-restore.outputs.cache-primary-key }} path: | tests/*.onnx tests/*.onnx.json tests/kaldi_model test-wheels: runs-on: ${{ matrix.os }} needs: - build-linux - build-windows - build-macos-arm - build-macos-intel - setup-tests-cache if: ${{ always() && true }} # Run even if some build jobs failed defaults: run: shell: bash strategy: fail-fast: false matrix: # https://docs.github.com/en/actions/reference/runners/github-hosted-runners#standard-github-hosted-runners-for-public-repositories # https://github.com/actions/runner-images os: [ubuntu-22.04, ubuntu-24.04, windows-2022, windows-2025, macos-14, macos-15, macos-15-intel, macos-26] # Status of Python versions (https://devguide.python.org/versions/) python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"] # 3.14 doesn't have piper-tts wheels yet steps: - uses: actions/checkout@v5 - name: Install just uses: taiki-e/install-action@just - name: Install uv and set the python version uses: astral-sh/setup-uv@v6 with: python-version: ${{ matrix.python-version }} - name: Set artifact name id: artifact-name run: | case "${{ matrix.os }}" in ubuntu-*) echo "name=wheels-linux" >> $GITHUB_OUTPUT ;; windows-*) echo "name=wheels-windows" >> $GITHUB_OUTPUT ;; macos-15-intel) echo "name=wheels-macos-intel" >> $GITHUB_OUTPUT ;; macos-*) echo "name=wheels-macos-arm" >> $GITHUB_OUTPUT ;; *) echo "Unexpected OS: ${{ matrix.os }}" 1>&2; exit 1 ;; esac - name: Download wheel artifacts uses: actions/download-artifact@v4 with: name: ${{ steps.artifact-name.outputs.name }} path: wheels/ continue-on-error: true # Continue even if no artifact found - name: Check wheels presence id: wheels-presence run: | shopt -s nullglob files=(wheels/*.whl) if (( ${#files[@]} > 0 )); then echo "found=true" >> $GITHUB_OUTPUT ls -l wheels/ else echo "found=false" >> $GITHUB_OUTPUT echo "No wheel artifacts found for ${{ matrix.os }}" 1>&2 fi - name: Restore cached tests setup uses: actions/cache/restore@v4 with: key: tests-setup-${{ hashFiles('Justfile') }}-v1 path: | tests/*.onnx tests/*.onnx.json tests/kaldi_model fail-on-cache-miss: true - name: Run tests if: steps.wheels-presence.outputs.found == 'true' # Must ignore warnings for piper-tts on Github Actions Windows runners run: | # just test-package -v -W 'ignore:Unsupported Windows version (2022server). ONNX Runtime supports Windows 10 and above, only.:UserWarning' -W 'ignore:Unsupported Windows version (2025server). ONNX Runtime supports Windows 10 and above, only.:UserWarning' just test-package-separately -W 'ignore:Unsupported Windows version (2022server). ONNX Runtime supports Windows 10 and above, only.:UserWarning' -W 'ignore:Unsupported Windows version (2025server). ONNX Runtime supports Windows 10 and above, only.:UserWarning' ================================================ FILE: .gitignore ================================================ _cmake_test_compile/ kaldi_active_grammar/exec examples/*.fst portable/ tests/*.onnx tests/*.onnx.json tests/**/*.wav tmp/ wheels */kaldi_model*/ */kaldi_model*.zip # general things to ignore venv*/ build/ _skbuild/ dist/ wheelhouse/ wheels/ *.egg-info/ *.egg *.py[cod] __pycache__/ *.so *~ .vscode/ pip-wheel-metadata/ # due to using tox and pytest .tox .cache .coverage ================================================ FILE: AGENTS.md ================================================ # Kaldi Active Grammar - Agent Information This document provides technical architectural information for AI coding agents (or humans!) working with the kaldi-active-grammar project. WARNING: This file may be auto-generated and/or out of date! ## Project Overview **Kaldi Active Grammar** is a Python package that enables context-based command and control using the Kaldi automatic speech recognition engine with dynamically manageable grammars. ### Key Technologies - **Speech Recognition**: Kaldi ASR engine - **Language**: Python 3.6+ - **Supported Platforms**: Windows, Linux, macOS (64-bit) - **Primary Integration**: Dragonfly speech recognition framework - **Model Architecture**: Kaldi nnet3 chain models ### Version Information - **Current Version**: See [`kaldi_active_grammar/__init__.py:8`](kaldi_active_grammar/__init__.py:8) - **Required Model Version**: See [`kaldi_active_grammar/__init__.py:10`](kaldi_active_grammar/__init__.py:10) - **Version History**: See [`CHANGELOG.md`](CHANGELOG.md:1) ## Core Modules ### Compiler (`kaldi_active_grammar/compiler.py`) The **Compiler** module is responsible for compiling grammar rules into FST (Finite State Transducer) format for use by the Kaldi decoder. **Key Classes:** - `Compiler`: Main compilation engine that manages grammar compilation and FST generation - `KaldiRule`: Represents an individual grammar rule with associated FST representation **Responsibilities:** - Grammar-to-FST compilation - Rule caching and management - Dynamic grammar loading/unloading - Lexicon handling and pronunciation generation ### Model (`kaldi_active_grammar/model.py`) The **Model** module manages the Kaldi acoustic model and lexicon operations. **Key Classes:** - `Lexicon`: Manages phoneme sets and phoneme conversion (CMU to XSAMPA) - `Model`: Orchestrates the acoustic model and lexicon **Responsibilities:** - Loading and validating Kaldi nnet3 chain models - Lexicon management (CMU, XSAMPA phoneme sets) - Word pronunciation generation (local via g2p_en or online) - Model version verification ### Wrapper (`kaldi_active_grammar/wrapper.py`) The **Wrapper** module provides the FFI (Foreign Function Interface) to native Kaldi binaries. **Key Classes:** - `KaldiAgfNNet3Decoder`: Main decoder for active grammar FSTs - `KaldiLafNNet3Decoder`: Alternative LAF (Linear Alignment Filter) decoder - `KaldiPlainNNet3Decoder`: Decoder for plain dictation **Responsibilities:** - Native library binding - Audio decoding - Hypothesis generation and lattice manipulation ### WFST (Weighted Finite State Transducer) (`kaldi_active_grammar/wfst.py`) The **WFST** module handles FST representation and manipulation. **Key Classes:** - `WFST`: Python-based FST implementation - `NativeWFST`: Native (C++-based) FST wrapper - `SymbolTable`: Maps symbols to numeric IDs for FST operations **Responsibilities:** - FST construction and modification - Symbol table management - FST serialization and caching ### Plain Dictation (`kaldi_active_grammar/plain_dictation.py`) The **PlainDictationRecognizer** module provides simple dictation recognition without grammar rules. **Features:** - Works with standard Kaldi HCLG.fst files - Fallback option for dictation-only use cases - Compatible with both pre-trained models and custom models ### Utilities (`kaldi_active_grammar/utils.py`) Utility functions for: - File discovery and path handling - Symbol table loading - External process management - Cross-platform compatibility ## Architecture Overview ``` ┌─────────────────────────────────────────┐ │ Dragonfly / User Application │ └────────────────┬────────────────────────┘ │ ┌─────────────────▼────────────────────────┐ │ Compiler (Grammar Rules → FSTs) │ ├──────────────────────────────────────────┤ │ • Grammar compilation │ │ • FST generation & caching │ │ • Rule management │ └────────────────┬────────────────────────┘ │ ┌─────────────────▼────────────────────────┐ │ Model (Acoustic Model + Lexicon) │ ├──────────────────────────────────────────┤ │ • Kaldi nnet3 chain model loading │ │ • Pronunciation generation │ │ • Lexicon management │ └────────────────┬────────────────────────┘ │ ┌─────────────────▼────────────────────────┐ │ Wrapper (FFI to Native Kaldi) │ ├──────────────────────────────────────────┤ │ • KaldiAgfNNet3Decoder │ │ • KaldiLafNNet3Decoder │ │ • Audio decoding & lattice generation │ └────────────────┬────────────────────────┘ │ ┌─────────────────▼────────────────────────┐ │ Native Kaldi Binaries (C++) │ ├──────────────────────────────────────────┤ │ • Acoustic model decoding │ │ • FST operations │ │ • Lattice operations │ └──────────────────────────────────────────┘ ``` ## Key Features & Capabilities ### Dynamic Grammar Management - Grammars can be marked active/inactive on a per-utterance basis - Enables context-aware command recognition - Improves accuracy by reducing possible recognitions ### Grammar Compilation - Multiple independent grammars with nonterminals - Separate compilation and dynamic stitching at decode-time - Shared dictation grammar between command grammars ### Performance & Accuracy - Context-based activation reduces vocabulary scope - Efficient FST-based representation - Support for weighted grammar rules ### Dictation Support - Integrated dictation grammar - Plain dictation interface (HCLG.fst compatible) - Pronunciation generation via g2p_en or online service ## Development Integration ### Testing - Test suite in `tests/` directory - Pytest configuration in `pyproject.toml` - Coverage reporting can be enabled - Integration tests for grammar compilation and decoding - Run tests with `just test` - To setup virtual environment for tests: `uv venv && uv pip install -r requirements-test.txt -r requirements-editable.txt` ### Examples - `examples/plain_dictation.py`: Plain dictation usage - `examples/mix_dictation.py`: Mixed command+dictation - `examples/full_example.py`: Comprehensive example - `examples/audio.py`: Audio handling utilities ### Build System - CMake-based native compilation - Scikit-build integration for wheel generation - Multi-platform support (Windows/Linux/macOS) - GitHub Actions CI/CD pipeline ## System Requirements - **Python**: 3.6+, 64-bit - **RAM**: 1GB+ (model + grammars) - **Disk Space**: 1GB+ (model + temporary files) - **Model Type**: Kaldi left-biphone nnet3 chain (specific modifications required) - **Audio Input**: Microphone or audio file ## Workflow 1. **Model Initialization**: Load Kaldi nnet3 chain model 2. **Grammar Definition**: Define command/rule grammar 3. **Compilation**: Compiler converts grammar rules to FSTs 4. **Activation**: Set which grammars are active for current utterance 5. **Decoding**: Wrapper processes audio through Kaldi decoder 6. **Recognition**: Return recognized utterance and associated action ================================================ FILE: CHANGELOG.md ================================================ # Changelog All notable changes to this project will be documented in this file. Note that the project (and python wheel) is built from a duorepo (2 separate repos used together), so changes from both will be reflected here, but the commits are spread between both. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) since v1.0.0. ## [3.2.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v3.2.0) - 2025-11-02 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v3.1.0...v3.2.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v3.1.0...kag-v3.2.0) ### Added * Comprehensive test suite with 80+ tests covering grammar compilation, plain dictation, and alternative dictation * Test infrastructure using pytest with TTS-generated test audio (Piper) * `AGENTS.md` documentation for AI coding agents with project architecture and development guidance * Exposed `NativeWFST` at package top-level for easier importing * Support for testing with multiple platforms and Python versions (3.9-3.13) ### Changed * **CI/CD Improvements**: * Implemented comprehensive caching of native binaries by commit hash * Added caching of test setup data * Updated build workflow to run on all pushes and PRs * Modified macOS wheel builds to use delocate instead of ad-hoc manual library handling * Improved Linux wheel build with cleaner output and better caching * Updated CI to support latest GitHub Actions runners (Ubuntu 24.04, Windows 2025, macOS 13/15/26) * Moved tests into main build workflow for faster feedback * Added notices for built wheels in CI output * Relaxed Python package requirements version specifiers for better compatibility * Updated setup.py classifiers to include Python 3.11, 3.12, 3.13, 3.14 * Dropped Python 2 from wheel tag (py3 instead of py2.py3), as Python 2 is no longer supported * Improved comments and cleanup in Justfile ### Fixed * Updated CI workflows to properly handle latest runner environments * Fixed Linux build configuration and wrapper script * Cleaned up and standardized build processes across all platforms ### Development * Refactored test structure for better organization and maintainability * Added test generators for creating synthetic speech using Piper TTS and Google TTS * Added helper utilities for test fixtures and audio generation * Improved test coverage for edge cases (empty audio, garbage audio, very short/long audio) * Added tests for complex grammar patterns (diamond, cascade, hub-and-spoke, etc.) * Added comprehensive alternative dictation tests with mocking ## [3.1.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v3.1.0) - 2021-11-24 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v3.0.0...v3.1.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v3.0.0...kag-v3.1.0) ### Fixed * Fix updating of SymbolTable multiple times for new words, so that there is only one instance for a single Model. ### Changed * Only mark lexicon stale if it was successfully modified. * Removed deprecated CLI binaries from Windows build, reducing wheel size by ~65%. ## [3.0.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v3.0.0) - 2021-10-31 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v2.1.0...v3.0.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v2.1.0...kag-v3.0.0) ### Changed * Pronunciation generation for lexicon now better supports local mode (using the `g2p_en` package), which is now also the default mode. It is also preferred over the online mode (using CMU's web service), which is now disabled by default. See the Setup section of the README for details. The new models now include the data files for `g2p_en`. * `PlainDictation` output now discards any silence words from transcript. * `lattice_beam` default value reduced from `6.0` to `5.0`, to hopefully avoid occasional errors. * Removed deprecated CLI binaries from build for linux/mac. ### Fixed * Whitespace in the model path is once again handled properly (thanks [@matthewmcintire](https://github.com/matthewmcintire)). * `NativeWFST.has_path()` now handles loops. * Linux/Mac binaries are now more stripped. ## [2.1.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v2.1.0) - 2021-04-04 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v2.0.2...v2.1.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v2.0.2...kag-v2.1.0) ### Added * NativeWFST support for checking for impossible graphs (no successful path), which can then fail to compile. * Debugging info for NativeWFST. ### Changed * `lattice_beam` default value reduced from `8.0` to `6.0`, to hopefully avoid occasional errors. ### Fixed * Reloading grammars with NativeWFST. ## [2.0.2](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v2.0.2) - 2021-03-30 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v2.0.0...v2.0.2) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v2.0.0...kag-v2.0.2) ### Changed * Minor fix for OpenBLAS compilation for some architectures on linux/mac ## [2.0.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v2.0.0) - 2021-03-21 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v1.8.0...v2.0.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v1.8.0...kag-v2.0.0) ### Added * Native FST support, via direct wrapping of OpenFST, rather than Python text-format implementation * Eliminates grammar (G) FST compilation step * Internalized many graph construction steps, via direct use of native Kaldi/OpenFST functions, rather than invoking separate CLI processes * Eliminates need for many temporary files (FSTs, `.conf`s, etc) and pipes * Example usage for allowing mixing of free dictation with strict command phrases * Experimental support for "look ahead" graphs, as an alternative to full HCLG compilation * Experimental support for rescoring with CARPA LMs * Experimental support for rescoring with RNN LMs * Experimental support for "priming" RNNLM previous left context for each utterance ### Changed * OpenBLAS is now the default linear algebra library (rather than Intel MKL) on Linux/MacOS * Because it is open source and provides good performance on all hardware (including AMD) * Windows is more difficult for this, and will be implemented soon in a later release * Default `tmp_dir` is now set to `[model_dir]/cache.tmp` * `tmp_dir` is now optional, and only needed if caching compiled FSTs (or for certain framework/option combinations) * File cache is now stored at `[model_dir]/file_cache.json` * Optimized adding many new words to the lexicon, in many different grammars, all in one loading session: only rebuild `L_disambig.fst` once at the end. * External interfaces: `Compiler.__init__()`, decoding setup, etc. * Internal interfaces: wrappers, etc. * Major refactoring of C++ components, with a new inheritance hierarchy and configuration mechanism, making it easier to use and test features with and without "activity" * Many build changes ### Removed * Python 2.7 support: it may still work, but will not be a focus. * Google cloud speech-to-text removed, as an unneeded dependency. Alternative dictation is still supported as an option, via a callback to an external provider. ### Deprecated * Separate CLI Kaldi/OpenFST executables * Indirect AGF graph compilation (framework==`agf-indirect`) * Non-native FSTs * parsing_framework==`text` ## [1.8.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v1.8.0) - 2020-09-05 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v1.7.0...v1.8.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v1.7.0...kag-v1.8.0) ### Added * New speech models (should be better in general, and support new noise resistance) * Make failed AGF graph compilation save and output stderr upon failure automatically * Example of complete usage with a grammar and microphone audio * Various documentation ### Changed * Top FST now accepts various noise phones (if present in speech model), making it more resistant to noise * Cleanup error handling in compiler, supporting Dragonfly backend automatically printing excerpt of the Rule that failed ### Fixed * Mysterious windows newline bug in some environments ## [1.7.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v1.7.0) - 2020-08-01 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v1.6.2...v1.7.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v1.6.2...kag-v1.7.0) ### Added * Add automatic saving of text FST & compiled FST files with log level 5 ### Changed * Miscellaneous naming ### Fixed * Support compiling some complex grammars (Caster text manipulation), by simplifying during compilation (remove epsilons, and determinize) ## [1.6.2](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v1.6.2) - 2020-07-20 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v1.6.1...v1.6.2) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v1.6.1...kag-v1.6.2) ### Fixed * Add missing rnnlm library file in MacOS build ## [1.6.1](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v1.6.1) - 2020-07-19 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v1.6.0...v1.6.1) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v1.6.0...kag-v1.6.1) ### Changed * Windows wheels now only require the VS2017 (not VS2019) redistributables to be installed ## [1.6.0](https://github.com/daanzu/kaldi-active-grammar/releases/tag/v1.6.0) - 2020-07-11 - Changes: [KaldiAG](https://github.com/daanzu/kaldi-active-grammar/compare/v1.5.0...v1.6.0) [KaldiFork](https://github.com/daanzu/kaldi-fork-active-grammar/compare/kag-v1.5.0...kag-v1.6.0) ### Added * Can now pass configuration dict to `KaldiAgfNNet3Decoder`, `PlainDictationRecognizer` (without `HCLG.fst`). * Continuous Integration builds run on GitHub Actions for Windows (x64), MacOS (x64), Linux (x64). ### Changed * Refactor of passing configuration to initialization. * `PlainDictationRecognizer.decode_utterance` can take `chunk_size` parameter. * Smaller binaries: MacOS 11MB -> 7.6MB, Linux 21MB -> 18MB. ### Fixed * Confidence measurement in the presence of multiple, redundant rules. * Python3 int division bug for cloud dictation. ## Earlier versions See [GitHub releases notes](https://github.com/daanzu/kaldi-active-grammar/releases). ================================================ FILE: CMakeLists.txt ================================================ cmake_minimum_required(VERSION 3.13.0) project(kaldi_binaries) include(ExternalProject) include(ProcessorCount) ProcessorCount(NCPU) if(NOT NCPU EQUAL 0) set(MAKE_FLAGS -j${NCPU}) endif() set(DST ${PROJECT_SOURCE_DIR}/kaldi_active_grammar/exec) if ("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Darwin") set(DST ${DST}/macos/) elseif("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Linux") set(DST ${DST}/linux/) else() set(DST ${DST}/windows/) endif() set(BINARIES ) set(LIBRARIES src/lib/libkaldi-dragonfly${CMAKE_SHARED_LIBRARY_SUFFIX} ) if("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Windows") message(FATAL_ERROR "CMake build not supported on Windows") # FIXME: copy files? # https://cmake.org/cmake/help/latest/command/foreach.html # https://stackoverflow.com/questions/34799916/copy-file-from-source-directory-to-binary-directory-using-cmake endif() find_program(MAKE_EXE NAMES make gmake nmake) if(DEFINED ENV{INTEL_MKL_DIR}) # Default: INTEL_MKL_DIR=/opt/intel/mkl/ message("Compiling with MKL in: $ENV{INTEL_MKL_DIR}") set(KALDI_CONFIG_FLAGS --shared --static-math --use-cuda=no --mathlib=MKL --mkl-root=$ENV{INTEL_MKL_DIR}) set(MATHLIB_BUILD_COMMAND true) else() if(NOT DEFINED OPENBLAS_REF) if("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Darwin") # Need newer version to build on macOS set(OPENBLAS_REF "v0.3.30") else() set(OPENBLAS_REF "v0.3.13") endif() endif() message("Compiling with OpenBLAS git ref: ${OPENBLAS_REF}") set(KALDI_CONFIG_FLAGS --shared --static-math --use-cuda=no --mathlib=OPENBLAS) set(MATHLIB_BUILD_COMMAND cd tools && git clone -b ${OPENBLAS_REF} --single-branch https://github.com/OpenMathLib/OpenBLAS && ${MAKE_EXE} ${MAKE_FLAGS} -C OpenBLAS DYNAMIC_ARCH=1 TARGET=GENERIC USE_LOCKING=1 USE_THREAD=0 all && ${MAKE_EXE} ${MAKE_FLAGS} -C OpenBLAS PREFIX=install install && cd ..) endif() if(DEFINED ENV{KALDI_BRANCH}) set(KALDI_BRANCH $ENV{KALDI_BRANCH}) else() message(FATAL_ERROR "KALDI_BRANCH not set! Use 'origin/master'?") # set(KALDI_BRANCH "origin/master") endif() message("MAKE_EXE = ${MAKE_EXE}") message("PYTHON_EXECUTABLE = ${PYTHON_EXECUTABLE}") message("PYTHON_INCLUDE_DIR = ${PYTHON_INCLUDE_DIR}") message("PYTHON_LIBRARY = ${PYTHON_LIBRARY}") message("PYTHON_VERSION_STRING = ${PYTHON_VERSION_STRING}") message("SKBUILD = ${SKBUILD}") message("KALDI_BRANCH = ${KALDI_BRANCH}") message("CMAKE_CURRENT_SOURCE_DIR = ${CMAKE_CURRENT_SOURCE_DIR}") message("CMAKE_CURRENT_BINARY_DIR = ${CMAKE_CURRENT_BINARY_DIR}") # CXXFLAGS are set and exported in kaldi-configure-wrapper.sh if(NOT "${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Windows") set(STRIP_LIBS_COMMAND find src/lib tools/openfst/lib -name *${CMAKE_SHARED_LIBRARY_SUFFIX} | xargs strip) # set(STRIP_DST_COMMAND find ${DST} [[[other specifiers]]] | xargs strip) if("${CMAKE_HOST_SYSTEM_NAME}" STREQUAL "Darwin") list(APPEND STRIP_LIBS_COMMAND -x) # list(APPEND STRIP_DST_COMMAND -x) endif() # set(STRIP_LIBS_COMMAND true) set(STRIP_DST_COMMAND true) ExternalProject_Add(kaldi GIT_CONFIG advice.detachedHead=false GIT_REPOSITORY https://github.com/daanzu/kaldi-fork-active-grammar.git GIT_TAG ${KALDI_BRANCH} GIT_SHALLOW TRUE CONFIGURE_COMMAND sed -i.bak -e "s/status=0/exit 0/g" tools/extras/check_dependencies.sh && sed -i.bak -e "s/openfst_add_CXXFLAGS = -g -O2/openfst_add_CXXFLAGS = -g0 -O3/g" tools/Makefile && cp ${PROJECT_SOURCE_DIR}/building/kaldi-configure-wrapper.sh src/ BUILD_IN_SOURCE TRUE BUILD_COMMAND ${MATHLIB_BUILD_COMMAND} && cd tools && ${MAKE_EXE} && cd openfst && autoreconf && cd ../../src && bash ./kaldi-configure-wrapper.sh ./configure ${KALDI_CONFIG_FLAGS} && ${MAKE_EXE} ${MAKE_FLAGS} depend && ${MAKE_EXE} ${MAKE_FLAGS} dragonfly LIST_SEPARATOR " " INSTALL_COMMAND ${STRIP_LIBS_COMMAND} && mkdir -p ${DST} && cp ${BINARIES} ${LIBRARIES} ${DST} && ${STRIP_DST_COMMAND} ) endif() install(CODE "MESSAGE(\"Installed kaldi engine binaries.\")") ================================================ FILE: Justfile ================================================ set ignore-comments set positional-arguments docker_repo := 'daanzu/kaldi-fork-active-grammar-manylinux' piper_voice := 'en_US-ryan-low' kaldi_model_url := 'https://github.com/daanzu/kaldi-active-grammar/releases/download/v3.0.0/kaldi_model_daanzu_20211030-smalllm.zip' _default: just --list just --summary build-linux python='python3': mkdir -p _skbuild rm -rf kaldi_active_grammar/exec rm -rf _skbuild/*/cmake-build/ _skbuild/*/cmake-install/ _skbuild/*/setuptools/ # {{python}} -m pip install -r requirements-build.txt # MKL with INTEL_MKL_DIR=/opt/intel/mkl/ {{python}} setup.py bdist_wheel build-dockcross *args='': building/dockcross-manylinux2010-x64 bash building/build-wheel-dockcross.sh manylinux2010_x86_64 {{args}} setup-dockcross: docker run --rm dockcross/manylinux2010-x64:20210127-72b83fc > building/dockcross-manylinux2010-x64 && chmod +x building/dockcross-manylinux2010-x64 @# [ ! -e building/dockcross-manylinux2010-x64 ] && docker run --rm dockcross/manylinux2010-x64 > building/dockcross-manylinux2010-x64 && chmod +x building/dockcross-manylinux2010-x64 || true pip-install-develop: KALDIAG_BUILD_SKIP_NATIVE=1 pip3 install --user -e . # Setup an editable development environment on linux setup-linux-develop kaldi_root_dir: # Compile kaldi_root_dir with: env CXXFLAGS=-O2 ./configure --mkl-root=/home/daanzu/intel/mkl/ --shared --static-math mkdir -p kaldi_active_grammar/exec/linux/ ln -sr {{kaldi_root_dir}}/tools/openfst/bin/fstarcsort kaldi_active_grammar/exec/linux/ ln -sr {{kaldi_root_dir}}/tools/openfst/bin/fstcompile kaldi_active_grammar/exec/linux/ ln -sr {{kaldi_root_dir}}/tools/openfst/bin/fstinfo kaldi_active_grammar/exec/linux/ ln -sr {{kaldi_root_dir}}/src/fstbin/fstaddselfloops kaldi_active_grammar/exec/linux/ ln -sr {{kaldi_root_dir}}/src/dragonfly/libkaldi-dragonfly.so kaldi_active_grammar/exec/linux/ ln -sr {{kaldi_root_dir}}/src/dragonflybin/compile-graph-agf kaldi_active_grammar/exec/linux/ watch-windows-develop config='Release': bash -c "watchexec -v --no-ignore -w /mnt/c/Work/Speech/kaldi/kaldi-windows/kaldiwin_vs2019_MKL/x64/ cp /mnt/c/Work/Speech/kaldi/kaldi-windows/kaldiwin_vs2019_MKL/x64/{{config}}/kaldi-dragonfly.dll /mnt/c/Work/Speech/kaldi/kaldi-active-grammar/kaldi_active_grammar/exec/windows/" test-model model_dir: cd {{invocation_directory()}} && rm -rf kaldi_model kaldi_model.tmp && cp -rp {{model_dir}} kaldi_model trigger-build ref='master': gh workflow run build.yml --ref {{ref}} setup-tests: uv run --no-project --with-requirements requirements-test.txt -m piper.download_voices --debug --download-dir tests/ '{{piper_voice}}' cd tests && [ ! -e kaldi_model ] && curl -L -C - -o kaldi_model.zip '{{kaldi_model_url}}' && unzip -o kaldi_model.zip || true # Common args: --lf -k test *args='': uv run --no-project --with-requirements requirements-test.txt --with-requirements requirements-editable.txt -m pytest "$@" # Test package after building wheel into wheels/ directory. Runs tests from within tests/ directory to prevent importing kaldi_active_grammar from source tree test-package *args='': uv run -v --no-project --isolated --with-requirements ../requirements-test.txt --with kaldi-active-grammar --find-links wheels/ --directory tests/ -m pytest "$@" test-package-separately *args='': uv run -v --no-project --isolated --with-requirements ../requirements-test.txt --with kaldi-active-grammar --find-links wheels/ --directory tests/ run_each_test_separately.py "$@" ================================================ FILE: LICENSE.txt ================================================ GNU AFFERO GENERAL PUBLIC LICENSE Version 3, 19 November 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. Developers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software. A secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate. Many developers of free software are heartened and encouraged by the resulting cooperation. However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public. The GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community. It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version. An older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals. This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU Affero General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Remote Network Interaction; Use with the GNU General Public License. Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU Affero General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If your software can interact with users remotely through a computer network, you should also make sure that it provides a way for users to get its source. For example, if your program is a web application, its interface could display a "Source" link that leads users to an archive of the code. There are many ways you could offer source, and different solutions will be better for different programs; see section 13 for the specific requirements. You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU AGPL, see . ================================================ FILE: README.md ================================================ # Kaldi Active Grammar > **Python Kaldi speech recognition with grammars that can be set active/inactive dynamically at decode-time** > Python package developed to enable context-based command & control of computer applications, as in the [Dragonfly](https://github.com/dictation-toolbox/dragonfly) speech recognition framework, using the [Kaldi](https://github.com/kaldi-asr/kaldi) automatic speech recognition engine. [![PyPI - Version](https://img.shields.io/pypi/v/kaldi-active-grammar.svg)](https://pypi.python.org/pypi/kaldi-active-grammar/) [![PyPI - Wheel](https://img.shields.io/pypi/wheel/kaldi-active-grammar.svg)](https://pypi.python.org/pypi/kaldi-active-grammar/) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/kaldi-active-grammar.svg)](https://pypi.python.org/pypi/kaldi-active-grammar/) [![PyPI - Downloads](https://img.shields.io/pypi/dm/kaldi-active-grammar.svg?logo=python)](https://pypi.python.org/pypi/kaldi-active-grammar/) [![GitHub - Downloads](https://img.shields.io/github/downloads/daanzu/kaldi-active-grammar/total?logo=github)](https://github.com/daanzu/kaldi-active-grammar/releases) ![Maintenance](https://img.shields.io/maintenance/yes/2026) ![PyPI - Status](https://img.shields.io/pypi/status/kaldi-active-grammar) [![Build](https://github.com/daanzu/kaldi-active-grammar/actions/workflows/build.yml/badge.svg)](https://github.com/daanzu/kaldi-active-grammar/actions/workflows/build.yml) [![PyPI - License](https://img.shields.io/pypi/l/kaldi-active-grammar)](https://github.com/daanzu/kaldi-active-grammar?tab=AGPL-3.0-1-ov-file#readme) [![Gitter](https://img.shields.io/gitter/room/daanzu/kaldi-active-grammar)](https://app.gitter.im/#/room/#kaldi-active-grammar_community:gitter.im) [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-002991.svg?logo=paypal)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) Normally, Kaldi decoding graphs are **monolithic**, require **expensive up-front off-line** compilation, and are **static during decoding**. Kaldi's new grammar framework allows **multiple independent** grammars with nonterminals, to be compiled separately and **stitched together dynamically** at decode-time, but all the grammars are **always active** and capable of being recognized. This project extends that to allow each grammar/rule to be **independently marked** as active/inactive **dynamically** on a **per-utterance** basis (set at the beginning of each utterance). Dragonfly is then capable of activating **only the appropriate grammars for the current environment**, resulting in increased accuracy due to fewer possible recognitions. Furthermore, the dictation grammar can be **shared** between all the command grammars, which can be **compiled quickly** without needing to include large-vocabulary dictation directly. See the [Changelog](CHANGELOG.md) for the latest updates. ### Features * **Binaries:** The Python package **includes all necessary binaries** for decoding on **Windows/Linux/MacOS**. Available on [PyPI](https://pypi.org/project/kaldi-active-grammar/#files). * Binaries are generated from my [fork of Kaldi](https://github.com/daanzu/kaldi-fork-active-grammar), which is only intended to be used by kaldi-active-grammar directly, and not as a stand-alone library. * **Pre-trained model:** A compatible **general English Kaldi nnet3 chain model** is trained on **~3000** hours of open audio. Available under [project releases](https://github.com/daanzu/kaldi-active-grammar/releases). * [**Model info and comparison**](docs/models.md) * Improved models are under development. * **Plain dictation:** Do you just want to recognize plain dictation? Seems kind of boring, but okay! There is an [**interface for plain dictation** (see below)](#plain-dictation-interface), using either your specified `HCLG.fst` file, or KaldiAG's included pre-trained dictation model. * **Dragonfly/Caster:** A compatible [**backend for Dragonfly**](https://github.com/daanzu/dragonfly/tree/kaldi/dragonfly/engines/backend_kaldi) is under development in the `kaldi` branch of my fork, and has been merged as of Dragonfly **v0.15.0**. * See its [documentation](https://dragonfly2.readthedocs.io/en/latest/kaldi_engine.html), try out a [demo](https://github.com/dictation-toolbox/dragonfly/blob/master/dragonfly/examples/kaldi_demo.py), or use the [loader](https://github.com/dictation-toolbox/dragonfly/blob/master/dragonfly/examples/kaldi_module_loader_plus.py) to run all normal dragonfly scripts. * You can try it out easily on Windows using a **simple no-install package**: see [Getting Started](#getting-started) below. * [Caster](https://github.com/dictation-toolbox/Caster) is supported as of KaldiAG **v0.6.0** and Dragonfly **v0.16.1**. * **Bootstrapped** since v0.2: development of KaldiAG is done entirely using KaldiAG. ### Demo Video
[![Demo Video](docs/demo_video.png)](https://youtu.be/Qk1mGbIJx3s)
### Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-002991.svg?logo=paypal)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) ### Related Repositories * [daanzu/kaldi-grammar-simple](https://github.com/daanzu/kaldi-grammar-simple) * [daanzu/speech-training-recorder](https://github.com/daanzu/speech-training-recorder) * [daanzu/dragonfly_daanzu_tools](https://github.com/daanzu/dragonfly_daanzu_tools) * [kmdouglass/caster-kaldi](https://github.com/kmdouglass/homelab/tree/master/speech-recognition/toolchains/caster-kaldi): Docker image to run KaldiAG + Dragonfly + Caster inside a container on Linux, using the host's microphone. ## Getting Started Want to get started **quickly & easily on Windows**? Available under [project releases](https://github.com/daanzu/kaldi-active-grammar/releases): * **`kaldi-dragonfly-winpython`**: A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-dragonfly-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2. Just unzip and run! * **`kaldi-caster-winpython-dev`**: [*more recent development version*] A self-contained, portable, batteries-included (python & libraries & model) distribution of kaldi-active-grammar + dragonfly2 + caster. Just unzip and run! Otherwise... ### Setup **Requirements**: * Python 3.6+; *64-bit required!* * OS: Windows/Linux/MacOS all supported * Only supports Kaldi left-biphone models, specifically *nnet3 chain* models, with specific modifications * ~1GB+ disk space for model plus temporary storage and cache, depending on your grammar complexity * ~1GB+ RAM for model and grammars, depending on your model and grammar complexity **Installation**: 1. Download compatible generic English Kaldi nnet3 chain model from [project releases](https://github.com/daanzu/kaldi-active-grammar/releases). Unzip the model and pass the directory path to kaldi-active-grammar constructor. * Or use your own model. Standard Kaldi models must be converted to be usable. Conversion can be performed automatically, but this hasn't been fully implemented yet. 1. Install Python package, which includes necessary Kaldi binaries: * The easy way to use kaldi-active-grammar is as a backend to dragonfly, which makes it easy to define grammars and resultant actions. * For this, simply run `pip install 'dragonfly2[kaldi]'` to install all necessary packages. See the [dragonfly documentation for details on installation](https://dragonfly2.readthedocs.io/en/latest/kaldi_engine.html#setup), plus how to define grammars and actions. * Alternatively, if you only want to use it directly (via a more low level interface), you can just run `pip install kaldi-active-grammar` 1. To support automatic generation of pronunciations for unknown words (not in the lexicon), you have two choices: * Local generation: Install the `g2p_en` package with `pip install 'kaldi-active-grammar[g2p_en]'` * The necessary data files are now included in the latest speech models I released with `v3.0.0`. * Online/cloud generation: Install the `requests` package with `pip install 'kaldi-active-grammar[online]'` **AND** pass `allow_online_pronunciations=True` to `Compiler.add_word()` or `Model.add_word()` * If both are available, the former is preferentially used. ### Troubleshooting * Errors installing * Make sure you're using a 64-bit Python. * You should install via `pip install kaldi-active-grammar` (directly or indirectly), *not* `python setup.py install`, in order to get the required binaries. * Update your `pip` (to at least `19.0+`) by executing `python -m pip install --upgrade pip`, to support the required python binary wheel package. * Errors running * Windows: `The code execution cannot proceed because VCRUNTIME140.dll was not found.` (or similar) * You must install the VC2017+ redistributable from Microsoft: [download page](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads), [direct link](https://aka.ms/vs/16/release/vc_redist.x64.exe). (This is usually already installed globally by other programs.) * Try deleting the Kaldi model `.tmp` directory, and re-running. * Try deleting the Kaldi model directory itself, re-downloading and/or re-extracting it, and re-running. (Note: You may want to make a copy of your `user_lexicon.txt` file before deleting, to put in the new model directory.) * For reporting issues, try running with `import logging; logging.basicConfig(level=1)` at the top of your main/loader file to enable full debugging logging. ## Documentation Formal documentation is somewhat lacking currently. To see example usage, examine: * [**Plain dictation interface**](examples/plain_dictation.py): Set up recognizer for plain dictation; perform decoding on given `wav` file. * [**Full example**](examples/full_example.py): Set up grammar compiler & decoder; set up a rule; perform decoding on live, real-time audio from microphone. * [**Backend for Dragonfly**](https://github.com/daanzu/dragonfly/tree/kaldi/dragonfly/engines/backend_kaldi): Many advanced features and complex interactions. The KaldiAG API is fairly low level, but basically: you define a set of grammar rules, then send in audio data, along with a bit mask of which rules are active at the beginning of each utterance, and receive back the recognized rule and text. The easy way is to go through Dragonfly, which makes it easy to define the rules, contexts, and actions. ### Building * Recommendation: use the binary wheels distributed for all major platforms. * Significant work has gone into allowing you to avoid the many repo/dependency downloads, GBs of disk space, and vCPU-hours needed for building from scratch. * They are built in public by automated Continuous Integration run on GitHub Actions: [see manifest](.github/workflows/build.yml). * Alternatively, to build for use locally: * Linux/MacOS: 1. `python -m pip install -r requirements-build.txt` 1. `python setup.py bdist_wheel` (see [`CMakeLists.txt`](CMakeLists.txt) for details) * Windows: * Less easily generally automated * You can follow the steps for Continuous Integration run on GitHub Actions: see the `build-windows` section of [the manifest](.github/workflows/build.yml). * Note: the project (and python wheel) is built from a duorepo (2 separate repos used together): 1. This repo, containing the external interface and higher-level logic, written in Python. 1. [My fork of Kaldi](https://github.com/daanzu/kaldi-fork-active-grammar), containing the lower-level code, written in C++. ## Contributing Issues, suggestions, and feature requests are welcome & encouraged. Pull requests are considered, but project structure is in flux. Donations are appreciated to encourage development. [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-002991.svg?logo=paypal)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/donate-GitHub-EA4AAA.svg?logo=githubsponsors)](https://github.com/sponsors/daanzu) ## Author * David Zurow ([@daanzu](https://github.com/daanzu)) ## License This project is licensed under the GNU Affero General Public License v3 (AGPL-3.0-or-later). See the [LICENSE.txt file](LICENSE.txt) for details. If this license is problematic for you, please contact me. ## Acknowledgments * Based on and including code from [Kaldi ASR](https://github.com/kaldi-asr/kaldi), under the Apache-2.0 license. * Code from [OpenFST](http://www.openfst.org/) and [OpenFST port for Windows](https://github.com/kkm000/openfst), under the Apache-2.0 license. * [Intel Math Kernel Library](https://software.intel.com/en-us/mkl), copyright (c) 2018 Intel Corporation, under the [Intel Simplified Software License](https://software.intel.com/en-us/license/intel-simplified-software-license), currently only used for Windows build. ================================================ FILE: building/build-wheel-dockcross.sh ================================================ #!/usr/bin/env bash # This script builds a Python wheel for kaldi-active-grammar using dockcross, # and is to be RUN WITHIN THE DOCKCROSS CONTAINER. It optionally installs Intel # MKL if MKL_URL is provided, then builds the wheel and repairs it for the # specified platform using auditwheel. # # Usage: ./build-wheel-dockcross.sh [--skip-native] [MKL_URL] # - --skip-native: Skip the native build step # - WHEEL_PLAT: The platform tag for the wheel (e.g., manylinux2014_x86_64) # - KALDI_BRANCH: The Kaldi branch to use for building # - MKL_URL: Optional URL to download and install Intel MKL set -e -x PYTHON_EXE=/opt/python/cp38-cp38/bin/python # Parse optional arguments and filter them out SKIP_NATIVE=false ARGS=() while [[ $# -gt 0 ]]; do case $1 in --skip-native) SKIP_NATIVE=true shift ;; *) ARGS+=("$1") shift ;; esac done # Set positional arguments from filtered array set -- "${ARGS[@]}" # Parse required arguments WHEEL_PLAT=$1 KALDI_BRANCH=$2 MKL_URL=$3 if [ -z "$PYTHON_EXE" ] || [ -z "$WHEEL_PLAT" ] || [ -z "$KALDI_BRANCH" ]; then echo "ERROR: variable not set!" exit 1 fi if [ -n "$MKL_URL" ]; then pushd _skbuild wget --no-verbose --no-clobber $MKL_URL mkdir -p /tmp/mkl MKL_FILE=$(basename $MKL_URL) tar zxf $MKL_FILE -C /tmp/mkl --strip-components=1 sed -i.bak -e 's/ACCEPT_EULA=decline/ACCEPT_EULA=accept/g' -e 's/ARCH_SELECTED=ALL/ARCH_SELECTED=INTEL64/g' /tmp/mkl/silent.cfg sudo /tmp/mkl/install.sh --silent /tmp/mkl/silent.cfg rm -rf /tmp/mkl export INTEL_MKL_DIR="/opt/intel/mkl/" popd fi if [ "$SKIP_NATIVE" = true ]; then export KALDIAG_BUILD_SKIP_NATIVE=1 # Patch the native binaries restored from cache to work with auditwheel repair below; final result should be idempotent patchelf --force-rpath --set-rpath "$(pwd)/kaldi_active_grammar.libs" kaldi_active_grammar/exec/linux/libkaldi-dragonfly.so readelf -d kaldi_active_grammar/exec/linux/libkaldi-dragonfly.so | egrep 'NEEDED|RUNPATH|RPATH' # ldd kaldi_active_grammar/exec/linux/libkaldi-dragonfly.so # LD_DEBUG=libs ldd kaldi_active_grammar/exec/linux/libkaldi-dragonfly.so else # Clean in preparation for native build mkdir -p _skbuild rm -rf _skbuild/*/cmake-install/ _skbuild/*/setuptools/ rm -rf kaldi_active_grammar/exec fi KALDI_BRANCH=$KALDI_BRANCH $PYTHON_EXE setup.py bdist_wheel # ls -lR kaldi_active_grammar/exec/linux mkdir -p wheelhouse for whl in dist/*.whl; do unzip -l $whl auditwheel show $whl auditwheel repair $whl --plat $WHEEL_PLAT -w wheelhouse/ # auditwheel -v repair $whl --plat $WHEEL_PLAT -w wheelhouse/ done ================================================ FILE: building/dockcross-manylinux2010-x64 ================================================ #!/usr/bin/env bash DEFAULT_DOCKCROSS_IMAGE=dockcross/manylinux2010-x64:latest #------------------------------------------------------------------------------ # Helpers # err() { echo -e >&2 ERROR: $@\\n } die() { err $@ exit 1 } has() { # eg. has command update local kind=$1 local name=$2 type -t $kind:$name | grep -q function } #------------------------------------------------------------------------------ # Command handlers # command:update-image() { docker pull $FINAL_IMAGE } help:update-image() { echo Pull the latest $FINAL_IMAGE . } command:update-script() { if cmp -s <( docker run --rm $FINAL_IMAGE ) $0; then echo $0 is up to date else echo -n Updating $0 '... ' docker run --rm $FINAL_IMAGE > $0 && echo ok fi } help:update-image() { echo Update $0 from $FINAL_IMAGE . } command:update() { command:update-image command:update-script } help:update() { echo Pull the latest $FINAL_IMAGE, and then update $0 from that. } command:help() { if [[ $# != 0 ]]; then if ! has command $1; then err \"$1\" is not an dockcross command command:help elif ! has help $1; then err No help found for \"$1\" else help:$1 fi else cat >&2 < ENDHELP exit 1 fi } #------------------------------------------------------------------------------ # Option processing # special_update_command='' while [[ $# != 0 ]]; do case $1 in --) shift break ;; --args|-a) ARG_ARGS="$2" shift 2 ;; --config|-c) ARG_CONFIG="$2" shift 2 ;; --image|-i) ARG_IMAGE="$2" shift 2 ;; update|update-image|update-script) special_update_command=$1 break ;; -*) err Unknown option \"$1\" command:help exit ;; *) break ;; esac done # The precedence for options is: # 1. command-line arguments # 2. environment variables # 3. defaults # Source the config file if it exists DEFAULT_DOCKCROSS_CONFIG=~/.dockcross FINAL_CONFIG=${ARG_CONFIG-${DOCKCROSS_CONFIG-$DEFAULT_DOCKCROSS_CONFIG}} [[ -f "$FINAL_CONFIG" ]] && source "$FINAL_CONFIG" # Set the docker image FINAL_IMAGE=${ARG_IMAGE-${DOCKCROSS_IMAGE-$DEFAULT_DOCKCROSS_IMAGE}} # Handle special update command if [ "$special_update_command" != "" ]; then case $special_update_command in update) command:update exit $? ;; update-image) command:update-image exit $? ;; update-script) command:update-script exit $? ;; esac fi # Set the docker run extra args (if any) FINAL_ARGS=${ARG_ARGS-${DOCKCROSS_ARGS}} # Bash on Ubuntu on Windows UBUNTU_ON_WINDOWS=$([ -e /proc/version ] && grep -l Microsoft /proc/version || echo "") # MSYS, Git Bash, etc. MSYS=$([ -e /proc/version ] && grep -l MINGW /proc/version || echo "") if [ -z "$UBUNTU_ON_WINDOWS" -a -z "$MSYS" ]; then USER_IDS=(-e BUILDER_UID="$( id -u )" -e BUILDER_GID="$( id -g )" -e BUILDER_USER="$( id -un )" -e BUILDER_GROUP="$( id -gn )") fi # Change the PWD when working in Docker on Windows if [ -n "$UBUNTU_ON_WINDOWS" ]; then WSL_ROOT="/mnt/" CFG_FILE=/etc/wsl.conf if [ -f "$CFG_FILE" ]; then CFG_CONTENT=$(cat $CFG_FILE | sed -r '/[^=]+=[^=]+/!d' | sed -r 's/\s+=\s/=/g') eval "$CFG_CONTENT" if [ -n "$root" ]; then WSL_ROOT=$root fi fi HOST_PWD=`pwd -P` HOST_PWD=${HOST_PWD/$WSL_ROOT//} elif [ -n "$MSYS" ]; then HOST_PWD=$PWD HOST_PWD=${HOST_PWD/\//} HOST_PWD=${HOST_PWD/\//:\/} else HOST_PWD=$PWD [ -L $HOST_PWD ] && HOST_PWD=$(readlink $HOST_PWD) fi # Mount Additional Volumes if [ -z "$SSH_DIR" ]; then SSH_DIR="$HOME/.ssh" fi HOST_VOLUMES= if [ -e "$SSH_DIR" -a -z "$MSYS" ]; then HOST_VOLUMES+="-v $SSH_DIR:/home/$(id -un)/.ssh" fi #------------------------------------------------------------------------------ # Now, finally, run the command in a container # TTY_ARGS= tty -s && [ -z "$MSYS" ] && TTY_ARGS=-ti CONTAINER_NAME=dockcross_$RANDOM docker run $TTY_ARGS --name $CONTAINER_NAME \ -v "$HOST_PWD":/work \ $HOST_VOLUMES \ "${USER_IDS[@]}" \ $FINAL_ARGS \ $FINAL_IMAGE "$@" run_exit_code=$? # Attempt to delete container rm_output=$(docker rm -f $CONTAINER_NAME 2>&1) rm_exit_code=$? if [[ $rm_exit_code != 0 ]]; then if [[ "$CIRCLECI" == "true" ]] && [[ $rm_output == *"Driver btrfs failed to remove"* ]]; then : # Ignore error because of https://circleci.com/docs/docker-btrfs-error/ else echo "$rm_output" exit $rm_exit_code fi fi exit $run_exit_code ################################################################################ # # This image is not intended to be run manually. # # To create a dockcross helper script for the # dockcross/manylinux2010-x64:latest image, run: # # docker run --rm dockcross/manylinux2010-x64:latest > dockcross-manylinux2010-x64-latest # chmod +x dockcross-manylinux2010-x64-latest # # You may then wish to move the dockcross script to your PATH. # ################################################################################ ================================================ FILE: building/kaldi-configure-wrapper.sh ================================================ #!/usr/bin/env bash # We use this wrapper script to set CXXFLAGS in the environment before calling # kaldi configure, to avoid issues with setting environment variables in # commands called from cmake. set -e -x export CXXFLAGS="-O3 -g0 -ftree-vectorize" # -g0: Request debugging information and also use level to specify how much information. The default level is 2. # Level 0 produces no debug information at all. Thus, -g0 negates -g. # Execute all arguments exec "$@" ================================================ FILE: docs/models.md ================================================ # Speech Recognition Models [![Donate](https://img.shields.io/badge/donate-GitHub-pink.svg)](https://github.com/sponsors/daanzu) [![Donate](https://img.shields.io/badge/donate-Patreon-orange.svg)](https://www.patreon.com/daanzu) [![Donate](https://img.shields.io/badge/donate-PayPal-green.svg)](https://paypal.me/daanzu) [![Donate](https://img.shields.io/badge/preferred-GitHub-black.svg)](https://github.com/sponsors/daanzu) ## Available Models * For **kaldi-active-grammar** * [kaldi_model_daanzu_20211030-biglm](https://github.com/daanzu/kaldi-active-grammar/releases/download/v3.0.0/kaldi_model_daanzu_20211030-biglm.zip) (1.05 GB) * [kaldi_model_daanzu_20211030-mediumlm](https://github.com/daanzu/kaldi-active-grammar/releases/download/v3.0.0/kaldi_model_daanzu_20211030-mediumlm.zip) (651 MB) * [kaldi_model_daanzu_20211030-smalllm](https://github.com/daanzu/kaldi-active-grammar/releases/download/v3.0.0/kaldi_model_daanzu_20211030-smalllm.zip) (400 MB) * [kaldi_model_daanzu_20200905_1ep-biglm](https://github.com/daanzu/kaldi-active-grammar/releases/download/v1.8.0/kaldi_model_daanzu_20200905_1ep-biglm.zip) (1.05 GB) * [kaldi_model_daanzu_20200905_1ep-mediumlm](https://github.com/daanzu/kaldi-active-grammar/releases/download/v1.8.0/kaldi_model_daanzu_20200905_1ep-mediumlm.zip) (651 MB) * [kaldi_model_daanzu_20200905_1ep-smalllm](https://github.com/daanzu/kaldi-active-grammar/releases/download/v1.8.0/kaldi_model_daanzu_20200905_1ep-smalllm.zip) (400 MB) * [kaldi_model_daanzu_20200328_1ep-mediumlm](https://github.com/daanzu/kaldi-active-grammar/releases/download/v1.4.0/kaldi_model_daanzu_20200328_1ep-mediumlm.zip) (322 MB) * For **generic kaldi**, or [**vosk**](https://github.com/alphacep/vosk-api) * [vosk-model-en-us-daanzu-20200328](https://github.com/daanzu/kaldi-active-grammar/releases/download/v1.4.0/vosk-model-en-us-daanzu-20200328.zip) * [vosk-model-en-us-daanzu-20200328-lgraph](https://github.com/daanzu/kaldi-active-grammar/releases/download/v1.4.0/vosk-model-en-us-daanzu-20200328-lgraph.zip) * If you have trouble downloading, try using `wget --continue` ## Basic info for KaldiAG models * **Latency**: I have yet to do formal latency testing, but for command grammars, the latency between the end of the utterance (as determined by the Voice Activity Detector) and receiving the final recognition results is in the range of 10-20ms. ## General Comparison * Metric: [Word Error Rate (WER)](https://en.wikipedia.org/wiki/Word_error_rate) * Data sets: * [LibriSpeech](http://www.openslr.org/12) Test Clean * [Mozilla Common Voice](https://voice.mozilla.org/en/datasets) English Test * [TED-LIUM Release 3](https://www.openslr.org/51/) Legacy Test * TestSet1: my test set combining multiple sources * Speech Comm: test set from [Google's Speech Commands Dataset](http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz), consisting of short single-word commands **Note**: The tests on commands are not necessarily fair, because they were performed using a full dictation grammar, rather than a reduced command-specific grammar. This is a worst case scenario for accuracy; in practice, speaking commands would perform much more accurately. | Engine | LS Test Clean | CV4 Test | Ted3 Test | TestSet1 | Speech Comm | |:--------------------------------------------------:|:-------------:|:---------:|:---------:|:---------:|:-----------:| | KaldiAG dgesr2-f-1ep LibriSpeech LM | 4.77 | **30.91** | **12.98** | **10.16** | **11.67** | | vosk-model-en-us-aspire-0.2 [carpa] | 17.90 | 69.76 | | | 55.90 | | vosk-model-small-en-us-0.3 | 19.30 | | | | 45.57 | | Zamia LibriSpeech LM | **4.56** | 34.28 | | 10.34 | 30.16 | | Amazon Transcribe **\*\*** | 8.21% | | | | | | CMU PocketSphinx (0.1.15) **\*\*** | 31.82% | | | | | | Google Speech-to-Text **\*\*** | 12.23% | | | | | | Mozilla DeepSpeech (0.6.1) **\*\*** | 7.55% | | | | | | Picovoice Cheetah (v1.2.0) **\*\*** | 10.49% | | | | | | Picovoice Cheetah LibriSpeech LM (v1.2.0) **\*\*** | 8.25% | | | | | | Picovoice Leopard (v1.0.0) **\*\*** | 8.34% | | | | | | Picovoice Leopard LibriSpeech LM (v1.0.0) **\*\*** | 6.58% | | | | | **\*\***: not tested by me; from [Picovoice speech-to-text-benchmark](https://github.com/Picovoice/speech-to-text-benchmark#results) ## Fine tuning for individual speakers Fine tuning a generic model for an individual speaker can greatly increase accuracy, at the small cost of recording some training data from the speaker themself. This training data can be recorded specifically for training purposes, or it can be retained from normal use while using another model (or even another engine). ### David * Very difficult speech. | Model | David Commands (test set) | David Dictation (test set) | |:---------------------------------------------------------------------:|:-------------------------:|:--------------------------:| | KaldiAG dgesr-f-1ep generic | 84.94 | 70.59 | | KaldiAG dgesr-f-1ep fine tuned on ~34hr of mixed commands + dictation | 7.11 | 14.46 | | Custom model trained only on ~34hr of mixed commands + dictation | 10.04 | 10.29 | ### Shervin * Accented speech. * Shervin Commands: ~1 hour, ~4000 utterances. * Shervin Dictation: ~20 minutes, ~250 utterances. | Model | Shervin Commands | Shervin Dictation | |:---------------------------------------------------------------:|:----------------:|:-----------------:| | KaldiAG dgesr2-f-1ep generic | 46.98 | 9.21 | | KaldiAG dgesr2-f-1ep fine tuned on Shervin Commands + Dictation | 9.76 | 2.40 | ================================================ FILE: examples/audio.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # from __future__ import division, print_function import collections, itertools, logging, time, threading, wave from six import binary_type, print_ from six.moves import queue import sounddevice import webrtcvad _log = logging.getLogger("audio") class MicAudio(object): """Streams raw audio from microphone. Data is received in a separate thread, and stored in a buffer, to be read from.""" FORMAT = 'int16' SAMPLE_WIDTH = 2 SAMPLE_RATE = 16000 CHANNELS = 1 BLOCKS_PER_SECOND = 100 BLOCK_SIZE_SAMPLES = int(SAMPLE_RATE / float(BLOCKS_PER_SECOND)) # Block size in number of samples BLOCK_DURATION_MS = int(1000 * BLOCK_SIZE_SAMPLES // SAMPLE_RATE) # Block duration in milliseconds def __init__(self, callback=None, buffer_s=0, flush_queue=True, start=True, input_device=None, self_threaded=None, reconnect_callback=None): self.callback = callback if callback is not None else lambda in_data: self.buffer_queue.put(in_data, block=False) self.flush_queue = bool(flush_queue) self.input_device = input_device self.self_threaded = bool(self_threaded) if reconnect_callback is not None and not callable(reconnect_callback): _log.error("Invalid reconnect_callback not callable: %r", reconnect_callback) reconnect_callback = None self.reconnect_callback = reconnect_callback self.buffer_queue = queue.Queue(maxsize=(buffer_s * 1000 // self.BLOCK_DURATION_MS)) self.stream = None self.thread = None self.thread_cancelled = False self.device_info = None self._connect(start=start) def _connect(self, start=None): callback = self.callback def proxy_callback(in_data, frame_count, time_info, status): callback(bytes(in_data)) # Must copy data from temporary C buffer! self.stream = sounddevice.RawInputStream( samplerate=self.SAMPLE_RATE, channels=self.CHANNELS, dtype=self.FORMAT, blocksize=self.BLOCK_SIZE_SAMPLES, # latency=80, device=self.input_device, callback=proxy_callback if not self.self_threaded else None, ) if self.self_threaded: self.thread_cancelled = False self.thread = threading.Thread(target=self._reader_thread, args=(callback,)) self.thread.daemon = True self.thread.start() if start: self.start() device_info = sounddevice.query_devices(self.stream.device) hostapi_info = sounddevice.query_hostapis(device_info['hostapi']) _log.info("streaming audio from '%s' using %s: %i sample_rate, %i block_duration_ms, %i latency_ms", device_info['name'], hostapi_info['name'], self.stream.samplerate, self.BLOCK_DURATION_MS, int(self.stream.latency*1000)) self.device_info = device_info def _reader_thread(self, callback): while not self.thread_cancelled and self.stream and not self.stream.closed: if self.stream.active and self.stream.read_available >= self.stream.blocksize: in_data, overflowed = self.stream.read(self.stream.blocksize) # print('_reader_thread', read_available, len(in_data), overflowed, self.stream.blocksize) if overflowed: _log.warning("audio stream overflow") callback(bytes(in_data)) # Must copy data from temporary C buffer! else: time.sleep(0.001) def destroy(self): self.stream.close() def reconnect(self): # FIXME: flapping old_device_info = self.device_info self.thread_cancelled = True if self.thread: self.thread.join() self.thread = None self.stream.close() self._connect(start=True) if self.reconnect_callback is not None: self.reconnect_callback(self) if self.device_info != old_device_info: raise Exception("Audio reconnect could not reconnect to the same device") def start(self): self.stream.start() def stop(self): self.stream.stop() def read(self, nowait=False): """Return a block of audio data. If nowait==False, waits for a block if necessary; else, returns False immediately if no block is available.""" if self.stream or (self.flush_queue and not self.buffer_queue.empty()): if nowait: try: return self.buffer_queue.get_nowait() # Return good block if available except queue.Empty as e: return False # Queue is empty for now else: return self.buffer_queue.get() # Wait for a good block and return it else: return None # We are done def read_loop(self, callback): """Block looping reading, repeatedly passing a block of audio data to callback.""" for block in iter(self): callback(block) def iter(self, nowait=False): """Generator that yields all audio blocks from microphone.""" while True: block = self.read(nowait=nowait) if block is None: break yield block def __iter__(self): """Generator that yields all audio blocks from microphone.""" return self.iter() def get_wav_length_s(self, data): assert isinstance(data, binary_type) length_bytes = len(data) assert self.FORMAT == 'int16' length_samples = length_bytes / self.SAMPLE_WIDTH return (float(length_samples) / self.SAMPLE_RATE) def write_wav(self, filename, data): # _log.debug("write wav %s", filename) wf = wave.open(filename, 'wb') wf.setnchannels(self.CHANNELS) # wf.setsampwidth(self.pa.get_sample_size(FORMAT)) assert self.FORMAT == 'int16' wf.setsampwidth(self.SAMPLE_WIDTH) wf.setframerate(self.SAMPLE_RATE) wf.writeframes(data) wf.close() @staticmethod def print_list(): print_("") print_("LISTING ALL INPUT DEVICES SUPPORTED BY PORTAUDIO") print_("(any device numbers not shown are for output only)") print_("") devices = sounddevice.query_devices() print_(devices) # for i in range(0, pa.get_device_count()): # info = pa.get_device_info_by_index(i) # if info['maxInputChannels'] > 0: # microphone? or just speakers # print_("DEVICE #%d" % info['index']) # print_(" %s" % info['name']) # print_(" input channels = %d, output channels = %d, defaultSampleRate = %d" % # (info['maxInputChannels'], info['maxOutputChannels'], info['defaultSampleRate'])) # # print_(info) # try: # supports16k = pa.is_format_supported(16000, # sample rate # input_device = info['index'], # input_channels = info['maxInputChannels'], # input_format = pyaudio.paInt16) # except ValueError: # print_(" NOTE: 16k sampling not supported, configure pulseaudio to use this device") print_("") class VADAudio(MicAudio): """Filter & segment audio with voice activity detection.""" def __init__(self, aggressiveness=3, **kwargs): super(VADAudio, self).__init__(**kwargs) self.vad = webrtcvad.Vad(aggressiveness) def vad_collector(self, start_window_ms=150, start_padding_ms=100, end_window_ms=150, end_padding_ms=None, complex_end_window_ms=None, ratio=0.8, blocks=None, nowait=False, ): """Generator/coroutine that yields series of consecutive audio blocks comprising each phrase, separated by yielding a single None. Determines voice activity by ratio of blocks in window_ms. Uses a buffer to include window_ms prior to being triggered. Example: (block, ..., block, None, block, ..., block, None, ...) |----phrase-----| |----phrase-----| """ assert end_padding_ms == None, "end_padding_ms not supported yet" num_start_window_blocks = max(1, int(start_window_ms // self.BLOCK_DURATION_MS)) num_start_padding_blocks = max(0, int((start_padding_ms or 0) // self.BLOCK_DURATION_MS)) num_end_window_blocks = max(1, int(end_window_ms // self.BLOCK_DURATION_MS)) num_complex_end_window_blocks = max(1, int((complex_end_window_ms or end_window_ms) // self.BLOCK_DURATION_MS)) num_end_padding_blocks = max(0, int((end_padding_ms or 0) // self.BLOCK_DURATION_MS)) _log.debug("%s: vad_collector: num_start_window_blocks=%s num_end_window_blocks=%s num_complex_end_window_blocks=%s", self, num_start_window_blocks, num_end_window_blocks, num_complex_end_window_blocks) audio_reconnect_threshold_blocks = 5 audio_reconnect_threshold_time = 50 * self.BLOCK_DURATION_MS / 1000 ring_buffer = collections.deque(maxlen=max( (num_start_window_blocks + num_start_padding_blocks), (num_end_window_blocks + num_end_padding_blocks), (num_complex_end_window_blocks + num_end_padding_blocks), )) ring_buffer_recent_slice = lambda num_blocks: itertools.islice(ring_buffer, max(0, (len(ring_buffer) - num_blocks)), None) triggered = False in_complex_phrase = False num_empty_blocks = 0 last_good_block_time = time.time() if blocks is None: blocks = self.iter(nowait=nowait) for block in blocks: if block is False or block is None: # Bad/empty block num_empty_blocks += 1 if (num_empty_blocks >= audio_reconnect_threshold_blocks) and (time.time() - last_good_block_time >= audio_reconnect_threshold_time): _log.warning("%s: no good block received recently, so reconnecting audio", self) self.reconnect() num_empty_blocks = 0 last_good_block_time = time.time() in_complex_phrase = yield block else: # Good block num_empty_blocks = 0 last_good_block_time = time.time() is_speech = self.vad.is_speech(block, self.SAMPLE_RATE) if not triggered: # Between phrases ring_buffer.append((block, is_speech)) num_voiced = len([1 for (_, speech) in ring_buffer_recent_slice(num_start_window_blocks) if speech]) if num_voiced >= (num_start_window_blocks * ratio): # Start of phrase triggered = True for block, _ in ring_buffer_recent_slice(num_start_padding_blocks + num_start_window_blocks): # print('|' if is_speech else '.', end='') # print('|' if in_complex_phrase else '.', end='') in_complex_phrase = yield block # print('#', end='') ring_buffer.clear() else: # Ongoing phrase in_complex_phrase = yield block # print('|' if is_speech else '.', end='') # print('|' if in_complex_phrase else '.', end='') ring_buffer.append((block, is_speech)) num_unvoiced = len([1 for (_, speech) in ring_buffer_recent_slice(num_end_window_blocks) if not speech]) num_complex_unvoiced = len([1 for (_, speech) in ring_buffer_recent_slice(num_complex_end_window_blocks) if not speech]) if (not in_complex_phrase and num_unvoiced >= (num_end_window_blocks * ratio)) or \ (in_complex_phrase and num_complex_unvoiced >= (num_complex_end_window_blocks * ratio)): # End of phrase triggered = False in_complex_phrase = yield None # print('*') ring_buffer.clear() def debug_print_simple(self): print("block_duration_ms=%s" % self.BLOCK_DURATION_MS) for block in self.iter(nowait=False): is_speech = self.vad.is_speech(block, self.SAMPLE_RATE) print('|' if is_speech else '.', end='') def debug_loop(self, *args, **kwargs): audio_iter = self.vad_collector(*args, **kwargs) next(audio_iter) while True: block = audio_iter.send(False) ================================================ FILE: examples/full_example.py ================================================ import logging, time import kaldi_active_grammar logging.basicConfig(level=20) model_dir = None # Default tmp_dir = None # Default ##### Set up grammar compiler & decoder compiler = kaldi_active_grammar.Compiler(model_dir=model_dir, tmp_dir=tmp_dir) # compiler.fst_cache.invalidate() decoder = compiler.init_decoder() ##### Set up a rule rule = kaldi_active_grammar.KaldiRule(compiler, 'TestRule') fst = rule.fst # Construct grammar in a FST previous_state = fst.add_state(initial=True) for word in "i will order the".split(): state = fst.add_state() fst.add_arc(previous_state, state, word) if word == 'the': # 'the' is optional, so we also allow an epsilon (silent) arc fst.add_arc(previous_state, state, None) previous_state = state final_state = fst.add_state(final=True) for word in ['egg', 'bacon', 'sausage']: fst.add_arc(previous_state, final_state, word) fst.add_arc(previous_state, final_state, 'spam', weight=8) # 'spam' is much more likely fst.add_arc(final_state, previous_state, None) # Loop back, with an epsilon (silent) arc rule.compile() rule.load() ##### You could add many more rules... ##### Perform decoding on live, real-time audio from microphone from audio import VADAudio audio = VADAudio() audio_iterator = audio.vad_collector(nowait=True) print("Listening...") in_phrase = False for block in audio_iterator: if block is False: # No audio block available time.sleep(0.001) elif block is not None: if not in_phrase: # Start of phrase kaldi_rules_activity = [True] # A bool for each rule in_phrase = True else: # Ongoing phrase kaldi_rules_activity = None # Irrelevant decoder.decode(block, False, kaldi_rules_activity) output, info = decoder.get_output() print("Partial phrase: %r" % (output,)) recognized_rule, words, words_are_dictation_mask, in_dictation = compiler.parse_partial_output(output) else: # End of phrase decoder.decode(b'', True) output, info = decoder.get_output() expected_error_rate = info.get('expected_error_rate', float('nan')) confidence = info.get('confidence', float('nan')) recognized_rule, words, words_are_dictation_mask = compiler.parse_output(output) is_acceptable_recognition = bool(recognized_rule) parsed_output = ' '.join(words) print("End of phrase: eer=%.2f conf=%.2f%s, rule %s, %r" % (expected_error_rate, confidence, (" [BAD]" if not is_acceptable_recognition else ""), recognized_rule, parsed_output)) in_phrase = False ================================================ FILE: examples/mix_dictation.py ================================================ import kaldi_active_grammar if __name__ == '__main__': import util compiler, decoder = util.initialize() ##### Set up a rule mixing strict commands with free dictation rule = kaldi_active_grammar.KaldiRule(compiler, 'TestRule') fst = rule.fst dictation_nonterm = '#nonterm:dictation' end_nonterm = '#nonterm:end' # Optional preface previous_state = fst.add_state(initial=True) next_state = fst.add_state() fst.add_arc(previous_state, next_state, 'cap') fst.add_arc(previous_state, next_state, None) # Optionally skip, with an epsilon (silent) arc # Required free dictation previous_state = next_state extra_state = fst.add_state() next_state = fst.add_state() # These two arcs together (always use together) will recognize one or more words of free dictation (but not zero): fst.add_arc(previous_state, extra_state, dictation_nonterm) fst.add_arc(extra_state, next_state, None, end_nonterm) # Loop repetition, alternating between a group of alternatives and more free dictation previous_state = next_state next_state = fst.add_state() for word in ['period', 'comma', 'colon']: fst.add_arc(previous_state, next_state, word) extra_state = fst.add_state() next_state = fst.add_state() fst.add_arc(next_state, extra_state, dictation_nonterm) fst.add_arc(extra_state, next_state, None, end_nonterm) fst.add_arc(next_state, previous_state, None) # Loop back, with an epsilon (silent) arc fst.add_arc(previous_state, next_state, None) # Optionally skip, with an epsilon (silent) arc # Finish up final_state = fst.add_state(final=True) fst.add_arc(next_state, final_state, None) rule.compile() rule.load() # Decode if __name__ == '__main__': util.do_recognition(compiler, decoder) ================================================ FILE: examples/plain_dictation.py ================================================ import logging, sys, wave from kaldi_active_grammar import PlainDictationRecognizer # logging.basicConfig(level=10) recognizer = PlainDictationRecognizer() # Or supply non-default model_dir, tmp_dir, or fst_file filename = sys.argv[1] if len(sys.argv) > 1 else 'test.wav' wave_file = wave.open(filename, 'rb') data = wave_file.readframes(wave_file.getnframes()) output_str, info = recognizer.decode_utterance(data) print(repr(output_str), info) # -> 'it depends on the context' ================================================ FILE: examples/requirements_audio.txt ================================================ sounddevice==0.3.* webrtcvad-wheels==2.0.* ================================================ FILE: examples/util.py ================================================ import time import kaldi_active_grammar from audio import VADAudio def initialize(model_dir=None, tmp_dir=None, config={}): compiler = kaldi_active_grammar.Compiler(model_dir=model_dir, tmp_dir=tmp_dir) decoder = compiler.init_decoder(config=config) return (compiler, decoder) def do_recognition(compiler, decoder, print_partial=True, cap_dictation=True): audio = VADAudio() audio_iterator = audio.vad_collector(nowait=True) print("Listening...") in_phrase = False for block in audio_iterator: if block is False: # No audio block available time.sleep(0.001) elif block is not None: if not in_phrase: # Start of phrase kaldi_rules_activity = [True] # A bool for each rule in_phrase = True else: # Ongoing phrase kaldi_rules_activity = None # Irrelevant decoder.decode(block, False, kaldi_rules_activity) output, info = decoder.get_output() if print_partial: print("Partial phrase: %r" % (output,)) recognized_rule, words, words_are_dictation_mask, in_dictation = compiler.parse_partial_output(output) else: # End of phrase decoder.decode(b'', True) output, info = decoder.get_output() expected_error_rate = info.get('expected_error_rate', float('nan')) confidence = info.get('confidence', float('nan')) recognized_rule, words, words_are_dictation_mask = compiler.parse_output(output) is_acceptable_recognition = bool(recognized_rule) if cap_dictation: words = [(word.upper() if word_in_dictation else word) for (word, word_in_dictation) in zip(words, words_are_dictation_mask)] parsed_output = ' '.join(words) print("End of phrase: eer=%.2f conf=%.2f%s, rule %s, %r" % (expected_error_rate, confidence, (" [BAD]" if not is_acceptable_recognition else ""), recognized_rule, parsed_output)) in_phrase = False ================================================ FILE: kaldi_active_grammar/LICENSE.txt ================================================ GNU AFFERO GENERAL PUBLIC LICENSE Version 3, 19 November 2007 Copyright (C) 2007 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software. The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. Developers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software. A secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate. Many developers of free software are heartened and encouraged by the resulting cooperation. However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public. The GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community. It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version. An older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals. This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license. The precise terms and conditions for copying, distribution and modification follow. TERMS AND CONDITIONS 0. Definitions. "This License" refers to version 3 of the GNU Affero General Public License. "Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. "The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. A "covered work" means either the unmodified Program or a work based on the Program. To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. 1. Source Code. The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. The Corresponding Source for a work in source code form is that same work. 2. Basic Permissions. All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. 3. Protecting Users' Legal Rights From Anti-Circumvention Law. No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. 4. Conveying Verbatim Copies. You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. 5. Conveying Modified Source Versions. You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: a) The work must carry prominent notices stating that you modified it, and giving a relevant date. b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. 6. Conveying Non-Source Forms. You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. "Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. 7. Additional Terms. "Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or d) Limiting the use for publicity purposes of names of licensors or authors of the material; or e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. 8. Termination. You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. 9. Acceptance Not Required for Having Copies. You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. 10. Automatic Licensing of Downstream Recipients. Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. 11. Patents. A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. 12. No Surrender of Others' Freedom. If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. 13. Remote Network Interaction; Use with the GNU General Public License. Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph. Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License. 14. Revised Versions of this License. The Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU Affero General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation. If the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. 15. Disclaimer of Warranty. THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. Limitation of Liability. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 17. Interpretation of Sections 15 and 16. If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see . Also add information on how to contact you by electronic and paper mail. If your software can interact with users remotely through a computer network, you should also make sure that it provides a way for users to get its source. For example, if your program is a web application, its interface could display a "Source" link that leads users to an archive of the code. There are many ways you could offer source, and different solutions will be better for different programs; see section 13 for the specific requirements. You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU AGPL, see . ================================================ FILE: kaldi_active_grammar/__init__.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # _name = 'kaldi_active_grammar' __version__ = '3.2.0' # __dev_version__ = __version__ + '.dev0' REQUIRED_MODEL_VERSION = '0.5.0' import logging _log = logging.getLogger('kaldi') # _handler = logging.NullHandler() # _log.addHandler(_handler) class KaldiError(Exception): pass from .compiler import Compiler, KaldiRule from .wrapper import KaldiAgfNNet3Decoder, KaldiLafNNet3Decoder, KaldiPlainNNet3Decoder from .wfst import NativeWFST, WFST from .plain_dictation import PlainDictationRecognizer from .utils import disable_donation_message ================================================ FILE: kaldi_active_grammar/__main__.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # import logging, os.path, shutil from six import print_ from . import _name from .utils import debug_timer from .compiler import Compiler from .model import Model, convert_generic_model_to_agf def main(): import argparse parser = argparse.ArgumentParser(prog='python -m %s' % _name) parser.add_argument('-v', '--verbose', action='store_true') parser.add_argument('-m', '--model_dir') parser.add_argument('-t', '--tmp_dir') parser.add_argument('command', choices=[ 'compile_agf_dictation_graph', 'compile_plain_dictation_graph', 'convert_generic_model_to_agf', 'add_word', 'generate_lexicon_files', 'reset_user_lexicon', 'generate_words_relabeled_file', ]) # FIXME: helps # FIXME: subparsers? args, unknown = parser.parse_known_args() logging.basicConfig(level=5 if args.verbose else logging.INFO) if args.command == 'compile_agf_dictation_graph': compiler = Compiler(args.model_dir, args.tmp_dir) g_filename = unknown.pop(0) if unknown else None print_("Compiling dictation graph...") compiler.compile_agf_dictation_fst(g_filename=g_filename) if args.command == 'compile_plain_dictation_graph': compiler = Compiler(args.model_dir, args.tmp_dir) g_filename = unknown.pop(0) if unknown else None output_filename = unknown.pop(0) if unknown else None print_("Compiling plain dictation graph...") compiler.compile_plain_dictation_fst(g_filename=g_filename, output_filename=output_filename) if args.command == 'convert_generic_model_to_agf': # if not args.model_dir: parser.error("MODEL_DIR required for %s" % args.command) file = unknown[0] convert_generic_model_to_agf(file, args.model_dir) if args.command == 'add_word': word = unknown[0] phones = unknown[1].split() if len(unknown) >= 2 else None pronunciations = Model(args.model_dir).add_word(word, phones) for phones in pronunciations: print_("Added word %r: %r" % (word, ' '.join(phones))) if args.command == 'generate_lexicon_files': Model(args.model_dir).generate_lexicon_files() print_("Generated lexicon files") if args.command == 'reset_user_lexicon': Model(args.model_dir).reset_user_lexicon() print_("Reset user lexicon") if args.command == 'generate_words_relabeled_file': Model.generate_words_relabeled_file(*unknown) print_("Generated words_relabeled file") if __name__ == '__main__': main() ================================================ FILE: kaldi_active_grammar/compiler.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # import collections, copy, logging, multiprocessing, os, re, shlex, shutil, subprocess, threading import concurrent.futures from contextlib import contextmanager from io import open from six.moves import range, zip from . import _log, KaldiError from .utils import ExternalProcess, debug_timer, platform, show_donation_message from .wfst import WFST, NativeWFST, SymbolTable from .model import Model from .wrapper import KaldiAgfCompiler, KaldiAgfNNet3Decoder, KaldiLafNNet3Decoder import kaldi_active_grammar.defaults as defaults _log = _log.getChild('compiler') ######################################################################################################################## class KaldiRule(object): cls_lock = threading.Lock() def __init__(self, compiler, name, nonterm=True, has_dictation=None, is_complex=None): """ :param nonterm: bool whether rule represents a nonterminal in the active-grammar-fst (only False for the top FST?) """ self.compiler = compiler self.name = name self.nonterm = nonterm self.has_dictation = has_dictation self.is_complex = is_complex # id: matches "nonterm:rule__"; 0-based; can/will change due to rule unloading! self.id = int(self.compiler.alloc_rule_id() if nonterm else -1) if self.id > self.compiler._max_rule_id: raise KaldiError("KaldiRule id > compiler._max_rule_id") if self.id in self.compiler.kaldi_rule_by_id_dict: raise KaldiError("KaldiRule id already in use") if self.id >= 0: self.compiler.kaldi_rule_by_id_dict[self.id] = self # Private/protected self._fst_text = None self.compiled = False self.loaded = False self.reloading = False # KaldiRule is in the process of the reload contextmanager self.has_been_loaded = False # KaldiRule was loaded, then reload() was called & completed, and now it is not currently loaded, and load() we need to call the decoder's reload self.destroyed = False # KaldiRule must not be used/referenced anymore # Public self.fst = WFST() if not self.compiler.native_fst else NativeWFST() self.matcher = None self.active = True def __repr__(self): return "%s(%s, %s)" % (self.__class__.__name__, self.id, self.name) fst_cache = property(lambda self: self.compiler.fst_cache) decoder = property(lambda self: self.compiler.decoder) pending_compile = property(lambda self: (self in self.compiler.compile_queue) or (self in self.compiler.compile_duplicate_filename_queue)) pending_load = property(lambda self: self in self.compiler.load_queue) fst_wrapper = property(lambda self: self.fst if self.fst.native else self.filepath) filename = property(lambda self: self.fst.filename) @property def filepath(self): assert self.filename assert self.compiler.tmp_dir is not None return os.path.join(self.compiler.tmp_dir, self.filename) def compile(self, lazy=False, duplicate=None): if self.destroyed: raise KaldiError("Cannot use a KaldiRule after calling destroy()") if self.compiled: return self if self.fst.native: if not self.filename: self.fst.compute_hash(self.fst_cache.dependencies_hash) assert self.filename else: # Handle compiling text WFST to binary if not self._fst_text: # self.fst.normalize_weights() self._fst_text = self.fst.get_fst_text(fst_cache=self.fst_cache) assert self.filename # if 'dictation' in self._fst_text: _log.log(50, '\n '.join(["%s: FST text:" % self] + self._fst_text.splitlines())) # log _fst_text if self.compiler.cache_fsts and self.fst_cache.fst_is_current(self.filepath, touch=True): _log.debug("%s: Skipped FST compilation thanks to FileCache" % self) if self.compiler.decoding_framework == 'agf' and self.fst.native: self.fst.compiled_native_obj = NativeWFST.load_file(self.filepath) self.compiled = True return self else: if duplicate: _log.warning("%s was supposed to be a duplicate compile, but was not found in FileCache") if lazy: if not self.pending_compile: # Special handling for rules that are an exact content match (and hence hash/name) with another (different) rule already in the compile_queue if not any(self.filename == kaldi_rule.filename for kaldi_rule in self.compiler.compile_queue if self != kaldi_rule): self.compiler.compile_queue.add(self) else: self.compiler.compile_duplicate_filename_queue.add(self) return self return self.finish_compile() def finish_compile(self): # Must be thread-safe! with self.cls_lock: self.compiler.prepare_for_compilation() _log.log(15, "%s: Compiling %sstate/%sarc FST%s%s" % (self, self.fst.num_states, self.fst.num_arcs, (" (%dbyte)" % len(self._fst_text)) if self._fst_text else "", (" to " + self.filename) if self.filename else "")) assert self.fst.native or self._fst_text if _log.isEnabledFor(3): if self.fst.native: self.fst.write_file('tmp_G.fst') if _log.isEnabledFor(2): if self._fst_text: _log.log(2, '\n '.join(["%s: FST text:" % self] + self._fst_text.splitlines())) # log _fst_text elif self.fst.native: self.fst.print() try: if self.compiler.decoding_framework == 'agf': if self.fst.native: self.fst.compiled_native_obj = self.compiler._compile_agf_graph(compile=True, nonterm=self.nonterm, input_fst=self.fst, return_output_fst=True, output_filename=(self.filepath if self.compiler.cache_fsts else None)) else: self.compiler._compile_agf_graph(compile=True, nonterm=self.nonterm, input_text=self._fst_text, output_filename=self.filepath) self._fst_text = None # Free memory elif self.compiler.decoding_framework == 'laf': # self.compiler._compile_laf_graph(compile=True, nonterm=self.nonterm, input_text=self._fst_text, output_filename=self.filepath) # Keep self._fst_text, for adding directly later pass else: raise KaldiError("unknown compiler.decoding_framework") except Exception as e: raise KaldiError("Exception while compiling", self) # Return this KaldiRule inside exception self.compiled = True return self def load(self, lazy=False): if self.destroyed: raise KaldiError("Cannot use a KaldiRule after calling destroy()") if lazy or self.pending_compile: self.compiler.load_queue.add(self) return self assert self.compiled if self.has_been_loaded: # FIXME: why is this necessary? self._do_reloading() else: if self.compiler.decoding_framework == 'agf': grammar_fst_index = self.decoder.add_grammar_fst(self.fst if self.fst.native else self.filepath) elif self.compiler.decoding_framework == 'laf': grammar_fst_index = self.decoder.add_grammar_fst(self.fst) if self.fst.native else self.decoder.add_grammar_fst_text(self._fst_text) else: raise KaldiError("unknown compiler decoding_framework") assert self.id == grammar_fst_index, "add_grammar_fst allocated invalid grammar_fst_index %d != %d for %s" % (grammar_fst_index, self.id, self) self.loaded = True self.has_been_loaded = True return self def _do_reloading(self): if self.compiler.decoding_framework == 'agf': return self.decoder.reload_grammar_fst(self.id, (self.fst if self.fst.native else self.filepath)) elif self.compiler.decoding_framework == 'laf': assert self.fst.native return self.decoder.reload_grammar_fst(self.id, self.fst) if not self.fst.native: return self.decoder.reload_grammar_fst_text(self.id, self._fst_text) # FIXME: not implemented else: raise KaldiError("unknown compiler decoding_framework") @contextmanager def reload(self): """ Used for modifying a rule in place, e.g. ListRef. """ if self.destroyed: raise KaldiError("Cannot use a KaldiRule after calling destroy()") was_loaded = self.loaded self.reloading = True self.fst.clear() self._fst_text = None self.compiled = False self.loaded = False yield if self.compiled and was_loaded: if not self.loaded: # FIXME: how is this different from the branch of the if above in load()? self._do_reloading() self.loaded = True elif was_loaded: # must be not self.compiled (i.e. the compile during reloading was lazy) self.compiler.load_queue.add(self) self.reloading = False def destroy(self): """ Destructor. Unloads rule. The rule should not be used/referenced anymore after calling! """ if self.destroyed: return if self.loaded: self.decoder.remove_grammar_fst(self.id) assert self not in self.compiler.compile_queue assert self not in self.compiler.compile_duplicate_filename_queue assert self not in self.compiler.load_queue else: if self in self.compiler.compile_queue: self.compiler.compile_queue.remove(self) if self in self.compiler.compile_duplicate_filename_queue: self.compiler.compile_duplicate_filename_queue.remove(self) if self in self.compiler.load_queue: self.compiler.load_queue.remove(self) # Adjust other kaldi_rules ids down, if above self.id, then rebuild dict other_kaldi_rules = list(self.compiler.kaldi_rule_by_id_dict.values()) other_kaldi_rules.remove(self) for kaldi_rule in other_kaldi_rules: if kaldi_rule.id > self.id: kaldi_rule.id -= 1 self.compiler.kaldi_rule_by_id_dict = { kaldi_rule.id: kaldi_rule for kaldi_rule in other_kaldi_rules } self.compiler.free_rule_id() self.destroyed = True ######################################################################################################################## class Compiler(object): def __init__(self, model_dir=None, tmp_dir=None, alternative_dictation=None, framework='agf-direct', native_fst=True, cache_fsts=True): # Supported parameter combinations: # framework='agf-indirect' native_fst=False (original method) # framework='agf-direct' native_fst=False (no external CLI programs needed) # framework='agf-direct' native_fst=True (no external CLI programs needed; no cache/temp files used) # framework='laf' native_fst=False (no reloading supported) # framework='laf' native_fst=True (no reloading supported) show_donation_message() self._log = _log AGF_INTERNAL_COMPILATION = True if framework == 'agf-direct': framework = 'agf' AGF_INTERNAL_COMPILATION = True if framework == 'agf-indirect': framework = 'agf' AGF_INTERNAL_COMPILATION = False assert not native_fst, "AGF with NativeWFST not supported" assert cache_fsts, "AGF must cache FSTs" self.decoding_framework = framework assert self.decoding_framework in ('agf', 'laf') self.parsing_framework = 'token' assert self.parsing_framework in ('token', 'text') self.native_fst = bool(native_fst) self.cache_fsts = bool(cache_fsts) self.alternative_dictation = alternative_dictation tmp_dir_needed = bool(self.cache_fsts) self.model = Model(model_dir, tmp_dir, tmp_dir_needed=tmp_dir_needed) self._lexicon_files_stale = False if self.native_fst: NativeWFST.init_class( osymbol_table=self.model.words_table, isymbol_table=self.model.words_table if self.decoding_framework != 'laf' else SymbolTable(self.files_dict['words.relabeled.txt']), wildcard_nonterms=self.wildcard_nonterms) self._agf_compiler = self._init_agf_compiler() if AGF_INTERNAL_COMPILATION else None self.decoder = None self._num_kaldi_rules = 0 self._max_rule_id = 999 self.nonterminals = tuple(['#nonterm:dictation'] + ['#nonterm:rule%i' % i for i in range(self._max_rule_id + 1)]) words_set = frozenset(self.model.words_table.words) self._oov_word = '' if ('' in self.model.words_table) else None # FIXME: make this configurable, for different models self._silence_words = frozenset(['!SIL']) & words_set # FIXME: make this configurable, for different models self._noise_words = frozenset(['', '!SIL']) & words_set # FIXME: make this configurable, for different models self.kaldi_rule_by_id_dict = collections.OrderedDict() # maps KaldiRule.id -> KaldiRule self.compile_queue = set() # KaldiRule self.compile_duplicate_filename_queue = set() # KaldiRule; queued KaldiRules with a duplicate filename (and thus contents), so can skip compilation self.load_queue = set() # KaldiRule; must maintain same order as order of instantiation! def init_decoder(self, config=None, dictation_fst_file=None): if self.decoder: raise KaldiError("Decoder already initialized") if dictation_fst_file is None: dictation_fst_file = self.dictation_fst_filepath decoder_kwargs = dict(model_dir=self.model_dir, tmp_dir=self.tmp_dir, dictation_fst_file=dictation_fst_file, max_num_rules=self._max_rule_id+1, config=config) if self.decoding_framework == 'agf': top_fst_rule = self.compile_top_fst() decoder_kwargs.update(top_fst=top_fst_rule.fst_wrapper) self.decoder = KaldiAgfNNet3Decoder(**decoder_kwargs) elif self.decoding_framework == 'laf': self.decoder = KaldiLafNNet3Decoder(**decoder_kwargs) else: raise KaldiError("Invalid Compiler.decoding_framework: %r" % self.decoding_framework) return self.decoder exec_dir = property(lambda self: self.model.exec_dir) model_dir = property(lambda self: self.model.model_dir) tmp_dir = property(lambda self: self.model.tmp_dir) files_dict = property(lambda self: self.model.files_dict) fst_cache = property(lambda self: self.model.fst_cache) num_kaldi_rules = property(lambda self: self._num_kaldi_rules) lexicon_words = property(lambda self: self.model.words_table.word_to_id_map) _longest_word = property(lambda self: self.model.longest_word) _default_dictation_g_filepath = property(lambda self: os.path.join(self.model_dir, defaults.DEFAULT_DICTATION_G_FILENAME)) _dictation_fst_filepath = property(lambda self: os.path.join(self.model_dir, (defaults.DEFAULT_DICTATION_FST_FILENAME if self.decoding_framework == 'agf' else 'Gr.fst'))) # FIXME: generalize _plain_dictation_hclg_fst_filepath = property(lambda self: os.path.join(self.model_dir, defaults.DEFAULT_PLAIN_DICTATION_HCLG_FST_FILENAME)) def alloc_rule_id(self): id = self._num_kaldi_rules self._num_kaldi_rules += 1 return id def free_rule_id(self): id = self._num_kaldi_rules self._num_kaldi_rules -= 1 return id #################################################################################################################### # Methods for compiling graphs. def add_word(self, word, phones=None, lazy_compilation=False, allow_online_pronunciations=False): pronunciations = self.model.add_word(word, phones=phones, lazy_compilation=lazy_compilation, allow_online_pronunciations=allow_online_pronunciations) self._lexicon_files_stale = True # Only mark lexicon stale if it was successfully modified (not an exception) return pronunciations def prepare_for_compilation(self): if self._lexicon_files_stale: self.model.generate_lexicon_files() self.model.load_words() # FIXME: This re-loading from the words.txt file may be unnecessary now that we have/use NativeWFST + SymbolTable, but it's not clear if it's safe to remove it. self.decoder.load_lexicon() if self._agf_compiler: # TODO: Just update the necessary files in the config self._agf_compiler.destroy() self._agf_compiler = self._init_agf_compiler() self._lexicon_files_stale = False def _compile_laf_graph(self, input_text=None, input_filename=None, output_filename=None, **kwargs): # FIXME: documentation with debug_timer(self._log.debug, "laf graph compilation"): format_kwargs = dict(self.files_dict, **kwargs) if input_text and input_filename: raise KaldiError("_compile_laf_graph passed both input_text and input_filename") elif input_text: input = ExternalProcess.shell.echo(input_text.encode('utf-8')) elif input_filename: input = input_filename else: raise KaldiError("_compile_laf_graph passed neither input_text nor input_filename") compile_command = input format = ExternalProcess.get_list_formatter(format_kwargs) compile_command |= ExternalProcess.fstcompile(*format('--isymbols={words_txt}', '--osymbols={words_txt}')) # g_filename = output_filename.replace('.fst', '.G.fst') compile_command |= output_filename compile_command() # fstrelabel --relabel_ipairs=relabel G.fst | fstarcsort --sort_type=ilabel | fstconvert --fst_type=const > Gr.fst def _init_agf_compiler(self): format_kwargs = dict(self.files_dict) config = dict( tree_rxfilename = '{tree}', model_rxfilename = '{final_mdl}', lex_rxfilename = '{L_disambig_fst}', disambig_rxfilename = '{disambig_int}', word_syms_filename = '{words_txt}', ) config = { key: value.format(**format_kwargs) for (key, value) in config.items() } return KaldiAgfCompiler(config) def _compile_agf_graph(self, compile=False, nonterm=False, simplify_lg=True, input_text=None, input_filename=None, input_fst=None, output_filename=None, return_output_fst=False, **kwargs): """ :param compile: bool whether to compile FST (False if it has already been compiled, like importing dictation FST) :param nonterm: bool whether rule represents a nonterminal in the active-grammar-fst (only False for the top FST?) :param simplify_lg: bool whether to simplify LG (disambiguate, and more) (do for command grammars, but not for dictation graph!) """ # Must be thread-safe! # Possible combinations of (compile,nonterm): (True,True) (True,False) (False,True) # FIXME: documentation verbose_level = 3 if self._log.isEnabledFor(5) else 0 format_kwargs = dict(self.files_dict, input_filename=input_filename, output_filename=output_filename, verbose=verbose_level, **kwargs) format_kwargs.update(nonterm_phones_offset=self.model.nonterm_phones_offset) format_kwargs.update(words_nonterm_begin=self.model.nonterm_words_offset, words_nonterm_end=self.model.nonterm_words_offset+1) format_kwargs.update(simplify_lg=str(bool(simplify_lg)).lower()) if self._agf_compiler: # Internal-style (no external CLI programs) config = dict( nonterm_phones_offset = self.model.nonterm_phones_offset, disambig_rxfilename = '{disambig_int}', simplify_lg = simplify_lg, verbose = verbose_level, tree_rxfilename = '{tree}', model_rxfilename = '{final_mdl}', lex_rxfilename = '{L_disambig_fst}', word_syms_filename = '{words_txt}', ) if output_filename: config.update(hclg_wxfilename=output_filename) elif self._log.isEnabledFor(3): import datetime config.update(hclg_wxfilename=os.path.join(self.tmp_dir, datetime.datetime.now().isoformat().replace(':', '') + '.fst')) if nonterm: config.update(grammar_prepend_nonterm=self.model.nonterm_words_offset, grammar_append_nonterm=self.model.nonterm_words_offset+1) config = { key: value.format(**format_kwargs) if isinstance(value, str) else value for (key, value) in config.items() } if 1 != sum(int(i is not None) for i in [input_text, input_filename, input_fst]): raise KaldiError("must pass exactly one input") if input_text: return self._agf_compiler.compile_graph(config, grammar_fst_text=input_text, return_graph=return_output_fst) if input_filename: return self._agf_compiler.compile_graph(config, grammar_fst_file=input_filename, return_graph=return_output_fst) if input_fst: return self._agf_compiler.compile_graph(config, grammar_fst=input_fst, return_graph=return_output_fst) elif True: # Pipeline-style assert not input_fst if input_text and input_filename: raise KaldiError("_compile_agf_graph passed both input_text and input_filename") elif input_text: input = ExternalProcess.shell.echo(input_text.encode('utf-8')) elif input_filename: input = input_filename else: raise KaldiError("_compile_agf_graph passed neither input_text nor input_filename") compile_command = input format = ExternalProcess.get_list_formatter(format_kwargs) args = [] # if True: (input | ExternalProcess.fstcompile(*format('--isymbols={words_txt}', '--osymbols={words_txt}')) | ExternalProcess.fstinfo | 'stats.log+')() # if True: (ExternalProcess.shell.echo(input_text) | ExternalProcess.fstcompile(*format('--isymbols={words_txt}', '--osymbols={words_txt}')) | (output_filename+'-G'))() if compile: compile_command |= ExternalProcess.fstcompile(*format('--isymbols={words_txt}', '--osymbols={words_txt}')) if self._log.isEnabledFor(5): g_txt_filename = output_filename.replace('.fst', '.G.fst.txt') self._log.log(5, "Saving text grammar FST to %s", g_txt_filename) with open(g_txt_filename, 'wb') as f: shutil.copyfileobj(copy.deepcopy(compile_command.commands[0].get_opt('stdin')), f) g_filename = output_filename.replace('.fst', '.G.fst') self._log.log(5, "Saving compiled grammar FST to %s", g_filename) (copy.deepcopy(compile_command) | g_filename)() args.extend(['--arcsort-grammar']) if nonterm: args.extend(format('--grammar-prepend-nonterm={words_nonterm_begin}', '--grammar-append-nonterm={words_nonterm_end}')) args.extend(format( '--nonterm-phones-offset={nonterm_phones_offset}', '--read-disambig-syms={disambig_int}', '--simplify-lg={simplify_lg}', '--verbose={verbose}', '{tree}', '{final_mdl}', '{L_disambig_fst}', '-', '{output_filename}')) compile_command |= ExternalProcess.compile_graph_agf(*args, **ExternalProcess.get_debug_stderr_kwargs(self._log)) ExternalProcess.execute_command_safely(compile_command, self._log) # if True: (ExternalProcess.shell.echo('%s -> %s\n' % (len(input_text), get_time_spent())) | ExternalProcess.shell('cat') | 'stats.log+')() else: # CLI-style (deprecated!) assert not input_fst run = lambda cmd, **kwargs: run_subprocess(cmd, format_kwargs, "agf graph compilation step", format_kwargs_update=dict(input_filename=output_filename), **kwargs) if compile: run("{exec_dir}fstcompile --isymbols={words_txt} --osymbols={words_txt} {input_filename}.txt {output_filename}") # run("cp {input_filename} {output_filename}-G") if compile: run("{exec_dir}fstarcsort --sort_type=ilabel {input_filename} {output_filename}") if nonterm: run("{exec_dir}fstconcat {tmp_dir}nonterm_begin.fst {input_filename} {output_filename}") if nonterm: run("{exec_dir}fstconcat {input_filename} {tmp_dir}nonterm_end.fst {output_filename}") # run("cp {input_filename} {output_filename}-G") run("{exec_dir}compile-graph --nonterm-phones-offset={nonterm_phones_offset} --read-disambig-syms={disambig_int} --verbose={verbose}" + " {tree} {final_mdl} {L_disambig_fst} {input_filename} {output_filename}") def compile_plain_dictation_fst(self, g_filename=None, output_filename=None): if g_filename is None: g_filename = self._default_dictation_g_filepath if output_filename is None: output_filename = self._plain_dictation_hclg_fst_filepath verbose_level = 5 if self._log.isEnabledFor(5) else 0 format_kwargs = dict(self.files_dict, g_filename=g_filename, output_filename=output_filename, verbose=verbose_level) format = ExternalProcess.get_list_formatter(format_kwargs) args = format('--read-disambig-syms={disambig_int}', '--simplify-lg=false', '--verbose={verbose}', '{tree}', '{final_mdl}', '{L_disambig_fst}', '{g_filename}', '{output_filename}') compile_command = ExternalProcess.compile_graph_agf(*args, **ExternalProcess.get_debug_stderr_kwargs(self._log)) compile_command() def compile_agf_dictation_fst(self, g_filename=None): if g_filename is None: g_filename = self._default_dictation_g_filepath self._compile_agf_graph(input_filename=g_filename, output_filename=self._dictation_fst_filepath, nonterm=True, simplify_lg=False) # def _compile_base_fsts(self): # filepaths = [self.tmp_dir + filename for filename in ['nonterm_begin.fst', 'nonterm_end.fst']] # if all(self.fst_cache.is_current(filepath) for filepath in filepaths): # return # format_kwargs = dict(self.files_dict) # def run(cmd): subprocess.check_call(cmd.format(**format_kwargs), shell=True) # FIXME: unsafe shell? # if platform == 'windows': # else: # run("(echo 0 1 #nonterm_begin 0^& echo 1) | {exec_dir}fstcompile.exe --isymbols={words_txt} > {tmp_dir}nonterm_begin.fst") # run("(echo 0 1 #nonterm_end 0^& echo 1) | {exec_dir}fstcompile.exe --isymbols={words_txt} > {tmp_dir}nonterm_end.fst") # run("(echo 0 1 \\#nonterm_begin 0; echo 1) | {exec_dir}fstcompile --isymbols={words_txt} > {tmp_dir}nonterm_begin.fst") # run("(echo 0 1 \\#nonterm_end 0; echo 1) | {exec_dir}fstcompile --isymbols={words_txt} > {tmp_dir}nonterm_end.fst") # for filepath in filepaths: # self.fst_cache.add(filepath) def compile_top_fst(self): return self._build_top_fst(nonterms=['#nonterm:rule'+str(i) for i in range(self._max_rule_id + 1)], noise_words=self._noise_words).compile() def compile_top_fst_dictation_only(self): return self._build_top_fst(nonterms=['#nonterm:dictation'], noise_words=self._noise_words).compile() def _build_top_fst(self, nonterms, noise_words): kaldi_rule = KaldiRule(self, 'top', nonterm=False) fst = kaldi_rule.fst state_initial = fst.add_state(initial=True) state_final = fst.add_state(final=True) state_return = fst.add_state() for nonterm in nonterms: fst.add_arc(state_initial, state_return, nonterm) fst.add_arc(state_return, state_final, None, '#nonterm:end') if noise_words: for (state_from, state_to) in [ (state_initial, state_final), # (state_initial, state_initial), # FIXME: test this # (state_final, state_final), ]: for word in noise_words: fst.add_arc(state_from, state_to, word) return kaldi_rule def _get_dictation_fst_filepath(self): if os.path.exists(self._dictation_fst_filepath): return self._dictation_fst_filepath self._log.error("cannot find dictation fst: %s", self._dictation_fst_filepath) # FIXME: Fall back to universal dictation? dictation_fst_filepath = property(_get_dictation_fst_filepath) # def _construct_dictation_states(self, fst, src_state, dst_state, number=(1,None), words=None, start_weight=None): # """ # Matches `number` words. # :param number: (0,None) or (1,None) or (1,1), where None is infinity. # """ # # unweighted=0.01 # if words is None: words = self.lexicon_words # word_probs = self._lexicon_word_probs # backoff_state = fst.add_state() # fst.add_arc(src_state, backoff_state, None, weight=start_weight) # if number[0] == 0: # fst.add_arc(backoff_state, dst_state, None) # for word, prob in word_probs.items(): # state = fst.add_state() # fst.add_arc(backoff_state, state, word, weight=prob) # if number[1] == None: # fst.add_arc(state, backoff_state, None) # fst.add_arc(state, dst_state, None) def compile_universal_grammar(self, words=None): """recognizes any sequence of words""" kaldi_rule = KaldiRule(self, 'universal', nonterm=False) if words is None: words = self.lexicon_words fst = kaldi_rule.fst backoff_state = fst.add_state(initial=True, final=True) for word in words: # state = fst.add_state() # fst.add_arc(backoff_state, state, word) # fst.add_arc(state, backoff_state, None) fst.add_arc(backoff_state, backoff_state, word) kaldi_rule.compile() return kaldi_rule def process_compile_and_load_queues(self): # Allowing this gives us leeway elsewhere # for kaldi_rule in self.compile_queue: # if kaldi_rule.compiled: # self._log.warning("compile_queue has %s but it is already compiled", kaldi_rule) # for kaldi_rule in self.compile_duplicate_filename_queue: # if kaldi_rule.compiled: # self._log.warning("compile_duplicate_filename_queue has %s but it is already compiled", kaldi_rule) # for kaldi_rule in self.load_queue: # if kaldi_rule.loaded: # self._log.warning("load_queue has %s but it is already loaded", kaldi_rule) # Clean out obsolete entries self.compile_queue.difference_update([kaldi_rule for kaldi_rule in self.compile_queue if kaldi_rule.compiled]) self.compile_duplicate_filename_queue.difference_update([kaldi_rule for kaldi_rule in self.compile_duplicate_filename_queue if kaldi_rule.compiled]) self.load_queue.difference_update([kaldi_rule for kaldi_rule in self.load_queue if kaldi_rule.loaded]) if self.compile_queue or self.compile_duplicate_filename_queue or self.load_queue: with concurrent.futures.ThreadPoolExecutor(max_workers=multiprocessing.cpu_count()) as executor: results = executor.map(lambda kaldi_rule: kaldi_rule.finish_compile(), self.compile_queue) # Load pending rules that have already been compiled # for kaldi_rule in (self.load_queue - self.compile_queue - self.compile_duplicate_filename_queue): # kaldi_rule.load() # self.load_queue.remove(kaldi_rule) # Handle rules as they are completed (have been compiled) for kaldi_rule in results: assert kaldi_rule.compiled self.compile_queue.remove(kaldi_rule) # if kaldi_rule in self.load_queue: # kaldi_rule.load() # self.load_queue.remove(kaldi_rule) # Handle rules that were pending compile but were duplicate and so compiled by/for another rule. They should be in the cache now for kaldi_rule in list(self.compile_duplicate_filename_queue): kaldi_rule.compile(duplicate=True) assert kaldi_rule.compiled self.compile_duplicate_filename_queue.remove(kaldi_rule) # if kaldi_rule in self.load_queue: # kaldi_rule.load() # self.load_queue.remove(kaldi_rule) # Load rules in correct order for kaldi_rule in sorted(self.load_queue, key=lambda kr: kr.id): kaldi_rule.load() assert kaldi_rule.loaded self.load_queue.remove(kaldi_rule) #################################################################################################################### # Methods for recognition. def prepare_for_recognition(self): try: if self.compile_queue or self.compile_duplicate_filename_queue or self.load_queue: self.process_compile_and_load_queues() except KaldiError: raise except Exception: raise KaldiError("Exception while compiling/loading rules in prepare_for_recognition") finally: if self.fst_cache.dirty: self.fst_cache.save() wildcard_nonterms = ('#nonterm:dictation', '#nonterm:dictation_cloud') def parse_output_for_rule(self, kaldi_rule, output): """Can be used even when self.parsing_framework == 'token', only for mimic (which contains no nonterms).""" labels = kaldi_rule.fst.does_match(output.split(), wildcard_nonterms=self.wildcard_nonterms) self._log.log(5, "parse_output_for_rule(%s, %r) got %r", kaldi_rule, output, labels) if labels is False: return None words = [label for label in labels if not label.startswith('#nonterm:')] parsed_output = ' '.join(words) if parsed_output.lower() != output: self._log.error("parsed_output(%r).lower() != output(%r)" % (parsed_output, output)) return words alternative_dictation_regex = re.compile(r'(?<=#nonterm:dictation_cloud )(.*?)(?= #nonterm:end)') # lookbehind & lookahead assertions def parse_output(self, output, dictation_info_func=None): """ dictation_info_func: Optional but required for using alternative_dictation; expected to be (audio_data, wrapper::KaldiNNet3Decoder.get_word_align output). """ assert self.parsing_framework == 'token' self._log.debug("parse_output(%r)" % output) if (output == '') or (output in self._noise_words): return None, [], [] nonterm_token, _, parsed_output = output.partition(' ') assert nonterm_token.startswith('#nonterm:rule') kaldi_rule_id = int(nonterm_token[len('#nonterm:rule'):]) kaldi_rule = self.kaldi_rule_by_id_dict[kaldi_rule_id] if self.alternative_dictation and dictation_info_func and kaldi_rule.has_dictation and '#nonterm:dictation_cloud' in parsed_output: try: if callable(self.alternative_dictation): alternative_text_func = self.alternative_dictation else: raise TypeError("Invalid alternative_dictation value: %r" % self.alternative_dictation) audio_data, word_align = dictation_info_func() self._log.log(5, "alternative_dictation word_align: %s", word_align) words, times, lengths = list(zip(*word_align)) # Find start & end word-index & byte-offset of each alternative dictation span dictation_spans = [{ 'index_start': index, 'offset_start': time, 'index_end': words.index('#nonterm:end', index), 'offset_end': times[words.index('#nonterm:end', index)], } for index, (word, time, length) in enumerate(word_align) if word.startswith('#nonterm:dictation_cloud')] # If last dictation is at end of utterance, it should include rest of audio_data; else, it should include half of audio_data between dictation end and start of next word dictation_span = dictation_spans[-1] if dictation_span['index_end'] == len(word_align) - 1: dictation_span['offset_end'] = len(audio_data) else: next_word_time = times[dictation_span['index_end'] + 1] dictation_span['offset_end'] = (dictation_span['offset_end'] + next_word_time) // 2 def replace_dictation(matchobj: re.Match) -> str: orig_text = matchobj.group(1) dictation_span = dictation_spans.pop(0) dictation_audio = audio_data[dictation_span['offset_start'] : dictation_span['offset_end']] with debug_timer(self._log.debug, 'alternative_dictation call'): alternative_text = alternative_text_func(dictation_audio) self._log.debug("alternative_dictation: %.2fs audio -> %r", (0.5 * len(dictation_audio) / 16000), alternative_text) # FIXME: hardcoded sample_rate! # alternative_dictation.write_wav('test.wav', dictation_audio) return (alternative_text or orig_text) parsed_output = self.alternative_dictation_regex.sub(replace_dictation, parsed_output) except Exception as e: self._log.exception("Exception performing alternative dictation") words = [] words_are_dictation_mask = [] in_dictation = False for word in parsed_output.split(): if word.startswith('#nonterm:'): if word.startswith('#nonterm:dictation'): in_dictation = True elif in_dictation and word == '#nonterm:end': in_dictation = False else: words.append(word) words_are_dictation_mask.append(in_dictation) return kaldi_rule, words, words_are_dictation_mask def parse_partial_output(self, output): assert self.parsing_framework == 'token' self._log.log(3, "parse_partial_output(%r)", output) if (output == '') or (output in self._noise_words): return None, [], [], False nonterm_token, _, parsed_output = output.partition(' ') assert nonterm_token.startswith('#nonterm:rule') kaldi_rule_id = int(nonterm_token[len('#nonterm:rule'):]) kaldi_rule = self.kaldi_rule_by_id_dict[kaldi_rule_id] words = [] words_are_dictation_mask = [] in_dictation = False for word in parsed_output.split(): if word.startswith('#nonterm:'): if word.startswith('#nonterm:dictation'): in_dictation = True elif in_dictation and word == '#nonterm:end': in_dictation = False else: words.append(word) words_are_dictation_mask.append(in_dictation) return kaldi_rule, words, words_are_dictation_mask, in_dictation ######################################################################################################################## # Utility functions. def remove_words_in_words(words, remove_words_func): return [word for word in words if not remove_words_func(word)] def remove_words_in_text(text, remove_words_func): return ' '.join(word for word in text.split() if not remove_words_func(word)) def remove_nonterms_in_words(words): return remove_words_in_words(words, lambda word: word.startswith('#nonterm:')) def remove_nonterms_in_text(text): return remove_words_in_text(text, lambda word: word.startswith('#nonterm:')) def run_subprocess(cmd, format_kwargs, description=None, format_kwargs_update=None, **kwargs): with debug_timer(_log.debug, description or "description", False), open(os.devnull, 'wb') as devnull: output = None if _log.isEnabledFor(logging.DEBUG) else devnull args = shlex.split(cmd.format(**format_kwargs), posix=(platform != 'windows')) _log.log(5, "subprocess.check_call(%r)", args) subprocess.check_call(args, stdout=output, stderr=output, **kwargs) if format_kwargs_update: format_kwargs.update(format_kwargs_update) ================================================ FILE: kaldi_active_grammar/defaults.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # DEFAULT_MODEL_DIR = 'kaldi_model' FILE_CACHE_FILENAME = 'file_cache.json' DEFAULT_DICTATION_G_FILENAME = 'G.fst' DEFAULT_DICTATION_FST_FILENAME = 'Dictation.fst' DEFAULT_PLAIN_DICTATION_HCLG_FST_FILENAME = 'HCLG.fst' ================================================ FILE: kaldi_active_grammar/ffi.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # """ FFI classes for Kaldi """ import os, re from cffi import FFI from .utils import exec_dir, platform _ffi = FFI() _library_binary_path = os.path.join(exec_dir, dict(windows='kaldi-dragonfly.dll', linux='libkaldi-dragonfly.so', macos='libkaldi-dragonfly.dylib')[platform]) _c_source_ignore_regex = re.compile(r'(\b(extern|DRAGONFLY_API)\b)|("C")|(//.*$)', re.MULTILINE) # Pattern for extraneous stuff to be removed def encode(text): """ For C interop: encode unicode text -> binary utf-8. """ return text.encode('utf-8') def decode(binary): """ For C interop: decode binary utf-8 -> unicode text. """ return binary.decode('utf-8') class FFIObject(object): def __init__(self): self.init_ffi() @classmethod def init_ffi(cls): cls._lib = _ffi.init_once(cls._init_ffi, cls.__name__ + '._init_ffi') @classmethod def _init_ffi(cls): _ffi.cdef(_c_source_ignore_regex.sub(' ', cls._library_header_text)) return _ffi.dlopen(_library_binary_path) ================================================ FILE: kaldi_active_grammar/kaldi/COPYING ================================================ Update to legal notice, made Feb 2012, modified Sep 2013. We would like to clarify that we are using a convention where multiple names in the Apache copyright headers, for example // Copyright 2009-2012 Yanmin Qian Arnab Ghoshal // 2013 Vassil Panayotov does not signify joint ownership of copyright of that file, except in cases where all those names were present in the original release made in March 2011-- you can use the version history to work this out, if this matters to you. Instead, we intend that those contributors who later modified the file, agree to release their changes under the Apache license. The conventional way of signifying this is to duplicate the Apache headers at the top of each file each time a change is made by a different author, but this would quickly become impractical. Where the copyright header says something like: // Copyright 2013 Johns Hopkins University (author: Daniel Povey) it is because the individual who wrote the code was at that institution as an employee, so the copyright is owned by the university (and we will have checked that the contributions were in accordance with the open-source policies of the institutions concerned, including getting them vetted individually where necessary). From a legal point of view the copyright ownership is that of the institution concerned, and the (author: xxx) in parentheses is just informational, to identify the actual person who wrote the code, and is not intended to have any legal implications. In some cases, however, particularly early on, we just wrote the name of the university or company concerned, without the actual author's name in parentheses. If you see something like // Copyright 2009-2012 Arnab Ghoshal Microsoft Corporation it does not imply that Arnab was working for Microsoft, it is because someone else contributed to the file while working at Microsoft (this would be Daniel Povey, in fact, who was working at Microsoft Research at the outset of the project). The list of authors of each file is in an essentially arbitrary order, but is often chronological if they contributed in different years. The original legal notice is below. Note: we are continuing to modify it by adding the names of new contributors, but at any given time, the list may be out of date. --- Legal Notices Each of the files comprising Kaldi v1.0 have been separately licensed by their respective author(s) under the terms of the Apache License v 2.0 (set forth below). The source code headers for each file specifies the individual authors and source material for that file as well the corresponding copyright notice. For reference purposes only: A cumulative list of all individual contributors and original source material as well as the full text of the Apache License v 2.0 are set forth below. Individual Contributors (in alphabetical order) Mohit Agarwal Tanel Alumae Gilles Boulianne Lukas Burget Dogan Can Guoguo Chen Gaofeng Cheng Cisco Corporation Pavel Denisov Ilya Edrenkin Ewald Enzinger Joachim Fainberg Daniel Galvez Pegah Ghahremani Arnab Ghoshal Ondrej Glembek Go Vivace Inc. Allen Guo Hossein Hadian Lv Hang Mirko Hannemann Hendy Irawan Navdeep Jaitly Johns Hopkins University Shiyin Kang Kirill Katsnelson Tom Ko Danijel Korzinek Gaurav Kumar Ke Li Matthew Maciejewski Vimal Manohar Yajie Miao Microsoft Corporation Petr Motlicek Xingyu Na Vincent Nguyen Lucas Ondel Vassil Panayotov Vijayaditya Peddinti Phonexia s.r.o. Ondrej Platek Daniel Povey Yanmin Qian Ariya Rastrow Saarland University Omid Sadjadi Petr Schwarz Yiwen Shao Nickolay V. Shmyrev Jan Silovsky Eduardo Silva Peter Smit David Snyder Alexander Solovets Georg Stemmer Pawel Swietojanski Jan "Yenda" Trmal Albert Vernon Karel Vesely Yiming Wang Shinji Watanabe Minhua Wu Haihua Xu Hainan Xu Xiaohui Zhang Other Source Material This project includes a port and modification of materials from JAMA: A Java Matrix Package under the following notice: "This software is a cooperative product of The MathWorks and the National Institute of Standards and Technology (NIST) which has been released to the public domain." This notice and the original code is available at http://math.nist.gov/javanumerics/jama/ This project includes a modified version of code published in Malvar, H., "Signal processing with lapped transforms," Artech House, Inc., 1992. The current copyright holder, Henrique S. Malvar, has given his permission for the release of this modified version under the Apache License 2.0. This project includes material from the OpenFST Library v1.2.7 available at http://www.openfst.org and released under the Apache License v. 2.0. [OpenFst COPYING file begins here] Licensed under the Apache License, Version 2.0 (the "License"); you may not use these files except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Copyright 2005-2010 Google, Inc. [OpenFst COPYING file ends here] ------------------------------------------------------------------------- Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================ FILE: kaldi_active_grammar/kaldi/__init__.py ================================================ ================================================ FILE: kaldi_active_grammar/kaldi/augment_phones_txt.py ================================================ #!/usr/bin/env python3 import argparse import re import os import sys def get_args(): parser = argparse.ArgumentParser(description="""This script augments a phones.txt file (a phone-level symbol table) by adding certain special symbols relating to grammar support. See ../add_nonterminals.sh for context.""") parser.add_argument('input_phones_txt', type=str, help='Filename of input phones.txt file, to be augmented') parser.add_argument('nonterminal_symbols_list', type=str, help='Filename of a file containing a list of nonterminal ' 'symbols, one per line. E.g. #nonterm:contact_list') parser.add_argument('output_phones_txt', type=str, help='Filename of output ' 'phones.txt file. May be the same as input-phones-txt.') args = parser.parse_args() return args def read_phones_txt(filename): """Reads the phones.txt file in 'filename', returns a 2-tuple (lines, highest_symbol) where 'lines' is all the lines the phones.txt as a list of strings, and 'highest_symbol' is the integer value of the highest-numbered symbol in the symbol table. It is an error if the phones.txt is empty or mis-formatted.""" # The use of latin-1 encoding does not preclude reading utf-8. latin-1 # encoding means "treat words as sequences of bytes", and it is compatible # with utf-8 encoding as well as other encodings such as gbk, as long as the # spaces are also spaces in ascii (which we check). It is basically how we # emulate the behavior of python before python3. whitespace = re.compile("[ \t]+") with open(filename, 'r', encoding='latin-1') as f: lines = [line.strip(" \t\r\n") for line in f] highest_numbered_symbol = 0 for line in lines: s = whitespace.split(line) try: i = int(s[1]) if i > highest_numbered_symbol: highest_numbered_symbol = i except: raise RuntimeError("Could not interpret line '{0}' in file '{1}'".format( line, filename)) if s[0] == '#nonterm_bos': raise RuntimeError("It looks like the symbol table {0} already has nonterminals " "in it.".format(filename)) return lines, highest_numbered_symbol def read_nonterminals(filename): """Reads the user-defined nonterminal symbols in 'filename', checks that it has the expected format and has no duplicates, and returns the nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', '#nonterm:phone_number', ... ]. """ ans = [line.strip(" \t\r\n") for line in open(filename, 'r', encoding='latin-1')] if len(ans) == 0: raise RuntimeError("The file {0} contains no nonterminal symbols.".format(filename)) for nonterm in ans: if nonterm[:9] != '#nonterm:': raise RuntimeError("In file '{0}', expected nonterminal symbols to start with '#nonterm:', found '{1}'" .format(filename, nonterm)) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def write_phones_txt(orig_lines, highest_numbered_symbol, nonterminals, filename): """Writes updated phones.txt to 'filename'. 'orig_lines' is the original lines in the phones.txt file as a list of strings (without the newlines); highest_numbered_symbol is the highest numbered symbol in the original phones.txt; nonterminals is a list of strings like '#nonterm:foo'.""" with open(filename, 'w', encoding='latin-1') as f: for l in orig_lines: print(l, file=f) cur_symbol = highest_numbered_symbol + 1 for n in ['#nonterm_bos', '#nonterm_begin', '#nonterm_end', '#nonterm_reenter' ] + nonterminals: print("{0} {1}".format(n, cur_symbol), file=f) cur_symbol = cur_symbol + 1 def main(): args = get_args() (lines, highest_symbol) = read_phones_txt(args.input_phones_txt) nonterminals = read_nonterminals(args.nonterminal_symbols_list) write_phones_txt(lines, highest_symbol, nonterminals, args.output_phones_txt) if __name__ == '__main__': main() ================================================ FILE: kaldi_active_grammar/kaldi/augment_phones_txt_py2.py ================================================ #!/usr/bin/env python3 from __future__ import print_function import argparse import re import os import sys def get_args(): parser = argparse.ArgumentParser(description="""This script augments a phones.txt file (a phone-level symbol table) by adding certain special symbols relating to grammar support. See ../add_nonterminals.sh for context.""") parser.add_argument('input_phones_txt', type=str, help='Filename of input phones.txt file, to be augmented') parser.add_argument('nonterminal_symbols_list', type=str, help='Filename of a file containing a list of nonterminal ' 'symbols, one per line. E.g. #nonterm:contact_list') parser.add_argument('output_phones_txt', type=str, help='Filename of output ' 'phones.txt file. May be the same as input-phones-txt.') args = parser.parse_args() return args def read_phones_txt(filename): """Reads the phones.txt file in 'filename', returns a 2-tuple (lines, highest_symbol) where 'lines' is all the lines the phones.txt as a list of strings, and 'highest_symbol' is the integer value of the highest-numbered symbol in the symbol table. It is an error if the phones.txt is empty or mis-formatted.""" # The use of latin-1 encoding does not preclude reading utf-8. latin-1 # encoding means "treat words as sequences of bytes", and it is compatible # with utf-8 encoding as well as other encodings such as gbk, as long as the # spaces are also spaces in ascii (which we check). It is basically how we # emulate the behavior of python before python3. whitespace = re.compile("[ \t]+") with open(filename, 'r') as f: lines = [line.strip(" \t\r\n") for line in f] highest_numbered_symbol = 0 for line in lines: s = whitespace.split(line) try: i = int(s[1]) if i > highest_numbered_symbol: highest_numbered_symbol = i except: raise RuntimeError("Could not interpret line '{0}' in file '{1}'".format( line, filename)) if s[0] == '#nonterm_bos': raise RuntimeError("It looks like the symbol table {0} already has nonterminals " "in it.".format(filename)) return lines, highest_numbered_symbol def read_nonterminals(filename): """Reads the user-defined nonterminal symbols in 'filename', checks that it has the expected format and has no duplicates, and returns the nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', '#nonterm:phone_number', ... ]. """ ans = [line.strip(" \t\r\n") for line in open(filename, 'r')] if len(ans) == 0: raise RuntimeError("The file {0} contains no nonterminal symbols.".format(filename)) for nonterm in ans: if nonterm[:9] != '#nonterm:': raise RuntimeError("In file '{0}', expected nonterminal symbols to start with '#nonterm:', found '{1}'" .format(filename, nonterm)) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def write_phones_txt(orig_lines, highest_numbered_symbol, nonterminals, filename): """Writes updated phones.txt to 'filename'. 'orig_lines' is the original lines in the phones.txt file as a list of strings (without the newlines); highest_numbered_symbol is the highest numbered symbol in the original phones.txt; nonterminals is a list of strings like '#nonterm:foo'.""" with open(filename, 'wb') as f: for l in orig_lines: print(l, file=f) cur_symbol = highest_numbered_symbol + 1 for n in ['#nonterm_bos', '#nonterm_begin', '#nonterm_end', '#nonterm_reenter' ] + nonterminals: print("{0} {1}".format(n, cur_symbol), file=f) cur_symbol = cur_symbol + 1 def main(): args = get_args() (lines, highest_symbol) = read_phones_txt(args.input_phones_txt) nonterminals = read_nonterminals(args.nonterminal_symbols_list) write_phones_txt(lines, highest_symbol, nonterminals, args.output_phones_txt) if __name__ == '__main__': main() ================================================ FILE: kaldi_active_grammar/kaldi/augment_words_txt.py ================================================ #!/usr/bin/env python3 import argparse import os import sys import re def get_args(): parser = argparse.ArgumentParser(description="""This script augments a words.txt file (a word-level symbol table) by adding certain special symbols relating to grammar support. See ../add_nonterminals.sh for context, and augment_phones_txt.py.""") parser.add_argument('input_words_txt', type=str, help='Filename of input words.txt file, to be augmented') parser.add_argument('nonterminal_symbols_list', type=str, help='Filename of a file containing a list of nonterminal ' 'symbols, one per line. E.g. #nonterm:contact_list') parser.add_argument('output_words_txt', type=str, help='Filename of output ' 'words.txt file. May be the same as input-words-txt.') args = parser.parse_args() return args def read_words_txt(filename): """Reads the words.txt file in 'filename', returns a 2-tuple (lines, highest_symbol) where 'lines' is all the lines the words.txt as a list of strings, and 'highest_symbol' is the integer value of the highest-numbered symbol in the symbol table. It is an error if the words.txt is empty or mis-formatted.""" # The use of latin-1 encoding does not preclude reading utf-8. latin-1 # encoding means "treat words as sequences of bytes", and it is compatible # with utf-8 encoding as well as other encodings such as gbk, as long as the # spaces are also spaces in ascii (which we check). It is basically how we # emulate the behavior of python before python3. whitespace = re.compile("[ \t]+") with open(filename, 'r', encoding='latin-1') as f: lines = [line.strip(" \t\r\n") for line in f] highest_numbered_symbol = 0 for line in lines: s = whitespace.split(line) try: i = int(s[1]) if i > highest_numbered_symbol: highest_numbered_symbol = i except: raise RuntimeError("Could not interpret line '{0}' in file '{1}'".format( line, filename)) if s[0] in [ '#nonterm_begin', '#nonterm_end' ]: raise RuntimeError("It looks like the symbol table {0} already has nonterminals " "in it.".format(filename)) return lines, highest_numbered_symbol def read_nonterminals(filename): """Reads the user-defined nonterminal symbols in 'filename', checks that it has the expected format and has no duplicates, and returns the nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', '#nonterm:phone_number', ... ]. """ ans = [line.strip(" \t\r\n") for line in open(filename, 'r', encoding='latin-1')] if len(ans) == 0: raise RuntimeError("The file {0} contains no nonterminal symbols.".format(filename)) for nonterm in ans: if nonterm[:9] != '#nonterm:': raise RuntimeError("In file '{0}', expected nonterminal symbols to start with '#nonterm:', found '{1}'" .format(filename, nonterm)) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def write_words_txt(orig_lines, highest_numbered_symbol, nonterminals, filename): """Writes updated words.txt to 'filename'. 'orig_lines' is the original lines in the words.txt file as a list of strings (without the newlines); highest_numbered_symbol is the highest numbered symbol in the original words.txt; nonterminals is a list of strings like '#nonterm:foo'.""" with open(filename, 'w', encoding='latin-1') as f: for l in orig_lines: print(l, file=f) cur_symbol = highest_numbered_symbol + 1 for n in [ '#nonterm_begin', '#nonterm_end' ] + nonterminals: print("{0} {1}".format(n, cur_symbol), file=f) cur_symbol = cur_symbol + 1 def main(): args = get_args() (lines, highest_symbol) = read_words_txt(args.input_words_txt) nonterminals = read_nonterminals(args.nonterminal_symbols_list) write_words_txt(lines, highest_symbol, nonterminals, args.output_words_txt) if __name__ == '__main__': main() ================================================ FILE: kaldi_active_grammar/kaldi/augment_words_txt_py2.py ================================================ #!/usr/bin/env python3 from __future__ import print_function import argparse import os import sys import re def get_args(): parser = argparse.ArgumentParser(description="""This script augments a words.txt file (a word-level symbol table) by adding certain special symbols relating to grammar support. See ../add_nonterminals.sh for context, and augment_phones_txt.py.""") parser.add_argument('input_words_txt', type=str, help='Filename of input words.txt file, to be augmented') parser.add_argument('nonterminal_symbols_list', type=str, help='Filename of a file containing a list of nonterminal ' 'symbols, one per line. E.g. #nonterm:contact_list') parser.add_argument('output_words_txt', type=str, help='Filename of output ' 'words.txt file. May be the same as input-words-txt.') args = parser.parse_args() return args def read_words_txt(filename): """Reads the words.txt file in 'filename', returns a 2-tuple (lines, highest_symbol) where 'lines' is all the lines the words.txt as a list of strings, and 'highest_symbol' is the integer value of the highest-numbered symbol in the symbol table. It is an error if the words.txt is empty or mis-formatted.""" # The use of latin-1 encoding does not preclude reading utf-8. latin-1 # encoding means "treat words as sequences of bytes", and it is compatible # with utf-8 encoding as well as other encodings such as gbk, as long as the # spaces are also spaces in ascii (which we check). It is basically how we # emulate the behavior of python before python3. whitespace = re.compile("[ \t]+") with open(filename, 'r') as f: lines = [line.strip(" \t\r\n") for line in f] highest_numbered_symbol = 0 for line in lines: s = whitespace.split(line) try: i = int(s[1]) if i > highest_numbered_symbol: highest_numbered_symbol = i except: raise RuntimeError("Could not interpret line '{0}' in file '{1}'".format( line, filename)) if s[0] in [ '#nonterm_begin', '#nonterm_end' ]: raise RuntimeError("It looks like the symbol table {0} already has nonterminals " "in it.".format(filename)) return lines, highest_numbered_symbol def read_nonterminals(filename): """Reads the user-defined nonterminal symbols in 'filename', checks that it has the expected format and has no duplicates, and returns the nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', '#nonterm:phone_number', ... ]. """ ans = [line.strip(" \t\r\n") for line in open(filename, 'r')] if len(ans) == 0: raise RuntimeError("The file {0} contains no nonterminal symbols.".format(filename)) for nonterm in ans: if nonterm[:9] != '#nonterm:': raise RuntimeError("In file '{0}', expected nonterminal symbols to start with '#nonterm:', found '{1}'" .format(filename, nonterm)) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def write_words_txt(orig_lines, highest_numbered_symbol, nonterminals, filename): """Writes updated words.txt to 'filename'. 'orig_lines' is the original lines in the words.txt file as a list of strings (without the newlines); highest_numbered_symbol is the highest numbered symbol in the original words.txt; nonterminals is a list of strings like '#nonterm:foo'.""" with open(filename, 'wb') as f: for l in orig_lines: print(l, file=f) cur_symbol = highest_numbered_symbol + 1 for n in [ '#nonterm_begin', '#nonterm_end' ] + nonterminals: print("{0} {1}".format(n, cur_symbol), file=f) cur_symbol = cur_symbol + 1 def main(): args = get_args() (lines, highest_symbol) = read_words_txt(args.input_words_txt) nonterminals = read_nonterminals(args.nonterminal_symbols_list) write_words_txt(lines, highest_symbol, nonterminals, args.output_words_txt) if __name__ == '__main__': main() ================================================ FILE: kaldi_active_grammar/kaldi/make_lexicon_fst.py ================================================ #!/usr/bin/env python3 # Copyright 2018 Johns Hopkins University (author: Daniel Povey) # Apache 2.0. # see get_args() below for usage message. import argparse import os import sys import math import re # The use of latin-1 encoding does not preclude reading utf-8. latin-1 # encoding means "treat words as sequences of bytes", and it is compatible # with utf-8 encoding as well as other encodings such as gbk, as long as the # spaces are also spaces in ascii (which we check). It is basically how we # emulate the behavior of python before python3. # sys.stdout = open(1, 'w', encoding='latin-1', newline='\n', closefd=False) # sys.stderr = open(2, 'w', encoding='latin-1', newline='\n', closefd=False) def get_args(): parser = argparse.ArgumentParser(description="""This script creates the text form of a lexicon FST, to be compiled by fstcompile using the appropriate symbol tables (phones.txt and words.txt) . It will mostly be invoked indirectly via utils/prepare_lang.sh. The output goes to the stdout.""") parser.add_argument('--sil-phone', dest='sil_phone', type=str, help="""Text form of optional-silence phone, e.g. 'SIL'. See also the --silprob option.""") parser.add_argument('--sil-prob', dest='sil_prob', type=float, default=0.0, help="""Probability of silence between words (including at the beginning and end of word sequences). Must be in the range [0.0, 1.0]. This refers to the optional silence inserted by the lexicon; see the --silphone option.""") parser.add_argument('--sil-disambig', dest='sil_disambig', type=str, help="""Disambiguation symbol to disambiguate silence, e.g. #5. Will only be supplied if you are creating the version of L.fst with disambiguation symbols, intended for use with cyclic G.fst. This symbol was introduced to fix a rather obscure source of nondeterminism of CLG.fst, that has to do with reordering of disambiguation symbols and phone symbols.""") parser.add_argument('--left-context-phones', dest='left_context_phones', type=str, help="""Only relevant if --nonterminals is also supplied; this relates to grammar decoding (see http://kaldi-asr.org/doc/grammar.html or src/doc/grammar.dox). Format is a list of left-context phones, in text form, one per line. E.g. data/lang/phones/left_context_phones.txt""") parser.add_argument('--nonterminals', type=str, help="""If supplied, --left-context-phones must also be supplied. List of user-defined nonterminal symbols such as #nonterm:contact_list, one per line. E.g. data/local/dict/nonterminals.txt.""") parser.add_argument('lexiconp', type=str, help="""Filename of lexicon with pronunciation probabilities (normally lexiconp.txt), with lines of the form 'word prob p1 p2...', e.g. 'a 1.0 ay'""") args = parser.parse_args() return args def read_lexiconp(filename): """Reads the lexiconp.txt file in 'filename', with lines like 'word pron p1 p2 ...'. Returns a list of tuples (word, pron_prob, pron), where 'word' is a string, 'pron_prob', a float, is the pronunciation probability (which must be >0.0 and would normally be <=1.0), and 'pron' is a list of strings representing phones. An element in the returned list might be ('hello', 1.0, ['h', 'eh', 'l', 'ow']). """ ans = [] found_empty_prons = False found_large_pronprobs = False # See the comment near the top of this file, RE why we use latin-1. with open(filename, 'r', encoding='latin-1') as f: whitespace = re.compile("[ \t]+") for line in f: a = whitespace.split(line.strip(" \t\r\n")) if len(a) < 2: print("{0}: error: found bad line '{1}' in lexicon file {2} ".format( sys.argv[0], line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) word = a[0] if word == "": # This would clash with the epsilon symbol normally used in OpenFst. print("{0}: error: found as a word in lexicon file " "{1}".format(line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) try: pron_prob = float(a[1]) except: print("{0}: error: found bad line '{1}' in lexicon file {2}, 2nd field " "should be pron-prob".format(sys.argv[0], line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) prons = a[2:] if pron_prob <= 0.0: print("{0}: error: invalid pron-prob in line '{1}' of lexicon file {2} ".format( sys.argv[0], line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) if len(prons) == 0: found_empty_prons = True ans.append( (word, pron_prob, prons) ) if pron_prob > 1.0: found_large_pronprobs = True if found_empty_prons: print("{0}: warning: found at least one word with an empty pronunciation " "in lexicon file {1}.".format(sys.argv[0], filename), file=sys.stderr) if found_large_pronprobs: print("{0}: warning: found at least one word with pron-prob >1.0 " "in {1}".format(sys.argv[0], filename), file=sys.stderr) if len(ans) == 0: print("{0}: error: found no pronunciations in lexicon file {1}".format( sys.argv[0], filename), file=sys.stderr) sys.exit(1) return ans def write_nonterminal_arcs(start_state, loop_state, next_state, nonterminals, left_context_phones): """This function relates to the grammar-decoding setup, see kaldi-asr.org/doc/grammar.html. It is called from write_fst_no_silence and write_fst_silence, and writes to the stdout some extra arcs in the lexicon FST that relate to nonterminal symbols. See the section "Special symbols in L.fst, kaldi-asr.org/doc/grammar.html#grammar_special_l. start_state: the start-state of L.fst. loop_state: the state of high out-degree in L.fst where words leave and enter. next_state: the number from which this function can start allocating its own states. the updated value of next_state will be returned. nonterminals: the user-defined nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', ... ]. left_context_phones: a list of phones that may appear as left-context, e.g. ['a', 'ah', ... '#nonterm_bos']. """ shared_state = next_state next_state += 1 final_state = next_state next_state += 1 print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=start_state, dest=shared_state, phone='#nonterm_begin', word='#nonterm_begin', cost=0.0)) for nonterminal in nonterminals: print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=loop_state, dest=shared_state, phone=nonterminal, word=nonterminal, cost=0.0)) # this_cost equals log(len(left_context_phones)) but the expression below # better captures the meaning. Applying this cost to arcs keeps the FST # stochatic (sum-to-one, like an HMM), so that if we do weight pushing # things won't get weird. In the grammar-FST code when we splice things # together we will cancel out this cost, see the function CombineArcs(). this_cost = -math.log(1.0 / len(left_context_phones)) for left_context_phone in left_context_phones: print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=shared_state, dest=loop_state, phone=left_context_phone, word='', cost=this_cost)) # arc from loop-state to a final-state with #nonterm_end as ilabel and olabel print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=loop_state, dest=final_state, phone='#nonterm_end', word='#nonterm_end', cost=0.0)) print("{state}\t{final_cost}".format( state=final_state, final_cost=0.0)) return next_state def write_fst_no_silence(lexicon, nonterminals=None, left_context_phones=None): """Writes the text format of L.fst to the standard output. This version is for when --sil-prob=0.0, meaning there is no optional silence allowed. 'lexicon' is a list of 3-tuples (word, pron-prob, prons) as returned by read_lexiconp(). 'nonterminals', which relates to grammar decoding (see kaldi-asr.org/doc/grammar.html), is either None, or the user-defined nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', ... ]. 'left_context_phones', which also relates to grammar decoding, and must be supplied if 'nonterminals' is supplied is either None or a list of phones that may appear as left-context, e.g. ['a', 'ah', ... '#nonterm_bos']. """ loop_state = 0 next_state = 1 # the next un-allocated state, will be incremented as we go. for (word, pronprob, pron) in lexicon: cost = -math.log(pronprob) cur_state = loop_state for i in range(len(pron) - 1): print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=next_state, phone=pron[i], word=(word if i == 0 else ''), cost=(cost if i == 0 else 0.0))) cur_state = next_state next_state += 1 i = len(pron) - 1 # note: i == -1 if pron is empty. print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=loop_state, phone=(pron[i] if i >= 0 else ''), word=(word if i <= 0 else ''), cost=(cost if i <= 0 else 0.0))) if nonterminals is not None: next_state = write_nonterminal_arcs( loop_state, loop_state, next_state, nonterminals, left_context_phones) print("{state}\t{final_cost}".format( state=loop_state, final_cost=0.0)) def write_fst_with_silence(lexicon, sil_prob, sil_phone, sil_disambig, nonterminals=None, left_context_phones=None): """Writes the text format of L.fst to the standard output. This version is for when --sil-prob != 0.0, meaning there is optional silence 'lexicon' is a list of 3-tuples (word, pron-prob, prons) as returned by read_lexiconp(). 'sil_prob', which is expected to be strictly between 0.. and 1.0, is the probability of silence 'sil_phone' is the silence phone, e.g. "SIL". 'sil_disambig' is either None, or the silence disambiguation symbol, e.g. "#5". 'nonterminals', which relates to grammar decoding (see kaldi-asr.org/doc/grammar.html), is either None, or the user-defined nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', ... ]. 'left_context_phones', which also relates to grammar decoding, and must be supplied if 'nonterminals' is supplied is either None or a list of phones that may appear as left-context, e.g. ['a', 'ah', ... '#nonterm_bos']. """ assert sil_prob > 0.0 and sil_prob < 1.0 sil_cost = -math.log(sil_prob) no_sil_cost = -math.log(1.0 - sil_prob); start_state = 0 loop_state = 1 # words enter and leave from here sil_state = 2 # words terminate here when followed by silence; this state # has a silence transition to loop_state. next_state = 3 # the next un-allocated state, will be incremented as we go. print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=start_state, dest=loop_state, phone='', word='', cost=no_sil_cost)) print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=start_state, dest=sil_state, phone='', word='', cost=sil_cost)) if sil_disambig is None: print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=sil_state, dest=loop_state, phone=sil_phone, word='', cost=0.0)) else: sil_disambig_state = next_state next_state += 1 print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=sil_state, dest=sil_disambig_state, phone=sil_phone, word='', cost=0.0)) print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=sil_disambig_state, dest=loop_state, phone=sil_disambig, word='', cost=0.0)) for (word, pronprob, pron) in lexicon: pron_cost = -math.log(pronprob) cur_state = loop_state for i in range(len(pron) - 1): print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=next_state, phone=pron[i], word=(word if i == 0 else ''), cost=(pron_cost if i == 0 else 0.0))) cur_state = next_state next_state += 1 i = len(pron) - 1 # note: i == -1 if pron is empty. print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=loop_state, phone=(pron[i] if i >= 0 else ''), word=(word if i <= 0 else ''), cost=no_sil_cost + (pron_cost if i <= 0 else 0.0))) print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=sil_state, phone=(pron[i] if i >= 0 else ''), word=(word if i <= 0 else ''), cost=sil_cost + (pron_cost if i <= 0 else 0.0))) if nonterminals is not None: next_state = write_nonterminal_arcs( start_state, loop_state, next_state, nonterminals, left_context_phones) print("{state}\t{final_cost}".format( state=loop_state, final_cost=0.0)) def write_words_txt(orig_lines, highest_numbered_symbol, nonterminals, filename): """Writes updated words.txt to 'filename'. 'orig_lines' is the original lines in the words.txt file as a list of strings (without the newlines); highest_numbered_symbol is the highest numbered symbol in the original words.txt; nonterminals is a list of strings like '#nonterm:foo'.""" with open(filename, 'w', encoding='latin-1') as f: for l in orig_lines: print(l, file=f) cur_symbol = highest_numbered_symbol + 1 for n in [ '#nonterm_begin', '#nonterm_end' ] + nonterminals: print("{0} {1}".format(n, cur_symbol), file=f) cur_symbol = cur_symbol + 1 def read_nonterminals(filename): """Reads the user-defined nonterminal symbols in 'filename', checks that it has the expected format and has no duplicates, and returns the nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', '#nonterm:phone_number', ... ]. """ ans = [line.strip(" \t\r\n") for line in open(filename, 'r', encoding='latin-1')] if len(ans) == 0: raise RuntimeError("The file {0} contains no nonterminals symbols.".format(filename)) for nonterm in ans: if nonterm[:9] != '#nonterm:': raise RuntimeError("In file '{0}', expected nonterminal symbols to start with '#nonterm:', found '{1}'" .format(filename, nonterm)) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def read_left_context_phones(filename): """Reads, checks, and returns a list of left-context phones, in text form, one per line. Returns a list of strings, e.g. ['a', 'ah', ..., '#nonterm_bos' ]""" ans = [line.strip(" \t\r\n") for line in open(filename, 'r', encoding='latin-1')] if len(ans) == 0: raise RuntimeError("The file {0} contains no left-context phones.".format(filename)) whitespace = re.compile("[ \t]+") for s in ans: if len(whitespace.split(s)) != 1: raise RuntimeError("The file {0} contains an invalid line '{1}'".format(filename, s) ) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def is_token(s): """Returns true if s is a string and is space-free.""" if not isinstance(s, str): return False whitespace = re.compile("[ \t\r\n]+") split_str = whitespace.split(s); return len(split_str) == 1 and s == split_str[0] def main(args=None): if args is None: args = get_args() lexicon = read_lexiconp(args.lexiconp) if args.nonterminals is None: nonterminals, left_context_phones = None, None else: if args.left_context_phones is None: print("{0}: if --nonterminals is specified, --left-context-phones must also " "be specified".format(sys.argv[0])) sys.exit(1) nonterminals = read_nonterminals(args.nonterminals) left_context_phones = read_left_context_phones(args.left_context_phones) if args.sil_prob == 0.0: write_fst_no_silence(lexicon, nonterminals=nonterminals, left_context_phones=left_context_phones) else: # Do some checking that the options make sense. if args.sil_prob < 0.0 or args.sil_prob >= 1.0: print("{0}: invalid value specified --sil-prob={1}".format( sys.argv[0], args.sil_prob), file=sys.stderr) sys.exit(1) if not is_token(args.sil_phone): print("{0}: you specified --sil-prob={1} but --sil-phone is set " "to '{2}'".format(sys.argv[0], args.sil_prob, args.sil_phone), file=sys.stderr) sys.exit(1) if args.sil_disambig is not None and not is_token(args.sil_disambig): print("{0}: invalid value --sil-disambig='{1}' was specified." "".format(sys.argv[0], args.sil_disambig), file=sys.stderr) sys.exit(1) write_fst_with_silence(lexicon, args.sil_prob, args.sil_phone, args.sil_disambig, nonterminals=nonterminals, left_context_phones=left_context_phones) # (lines, highest_symbol) = read_words_txt(args.input_words_txt) # nonterminals = read_nonterminals(args.nonterminal_symbols_list) # write_words_txt(lines, highest_symbol, nonterminals, args.output_words_txt) if __name__ == '__main__': main() ================================================ FILE: kaldi_active_grammar/kaldi/make_lexicon_fst_py2.py ================================================ #!/usr/bin/env python3 # Copyright 2018 Johns Hopkins University (author: Daniel Povey) # Apache 2.0. # see get_args() below for usage message. from __future__ import print_function import argparse import os import sys import math import re # The use of latin-1 encoding does not preclude reading utf-8. latin-1 # encoding means "treat words as sequences of bytes", and it is compatible # with utf-8 encoding as well as other encodings such as gbk, as long as the # spaces are also spaces in ascii (which we check). It is basically how we # emulate the behavior of python before python3. # sys.stdout = open(1, 'w', encoding='latin-1', closefd=False) # sys.stderr = open(2, 'w', encoding='latin-1', closefd=False) def get_args(): parser = argparse.ArgumentParser(description="""This script creates the text form of a lexicon FST, to be compiled by fstcompile using the appropriate symbol tables (phones.txt and words.txt) . It will mostly be invoked indirectly via utils/prepare_lang.sh. The output goes to the stdout.""") parser.add_argument('--sil-phone', dest='sil_phone', type=str, help="""Text form of optional-silence phone, e.g. 'SIL'. See also the --silprob option.""") parser.add_argument('--sil-prob', dest='sil_prob', type=float, default=0.0, help="""Probability of silence between words (including at the beginning and end of word sequences). Must be in the range [0.0, 1.0]. This refers to the optional silence inserted by the lexicon; see the --silphone option.""") parser.add_argument('--sil-disambig', dest='sil_disambig', type=str, help="""Disambiguation symbol to disambiguate silence, e.g. #5. Will only be supplied if you are creating the version of L.fst with disambiguation symbols, intended for use with cyclic G.fst. This symbol was introduced to fix a rather obscure source of nondeterminism of CLG.fst, that has to do with reordering of disambiguation symbols and phone symbols.""") parser.add_argument('--left-context-phones', dest='left_context_phones', type=str, help="""Only relevant if --nonterminals is also supplied; this relates to grammar decoding (see http://kaldi-asr.org/doc/grammar.html or src/doc/grammar.dox). Format is a list of left-context phones, in text form, one per line. E.g. data/lang/phones/left_context_phones.txt""") parser.add_argument('--nonterminals', type=str, help="""If supplied, --left-context-phones must also be supplied. List of user-defined nonterminal symbols such as #nonterm:contact_list, one per line. E.g. data/local/dict/nonterminals.txt.""") parser.add_argument('lexiconp', type=str, help="""Filename of lexicon with pronunciation probabilities (normally lexiconp.txt), with lines of the form 'word prob p1 p2...', e.g. 'a 1.0 ay'""") args = parser.parse_args() return args def read_lexiconp(filename): """Reads the lexiconp.txt file in 'filename', with lines like 'word pron p1 p2 ...'. Returns a list of tuples (word, pron_prob, pron), where 'word' is a string, 'pron_prob', a float, is the pronunciation probability (which must be >0.0 and would normally be <=1.0), and 'pron' is a list of strings representing phones. An element in the returned list might be ('hello', 1.0, ['h', 'eh', 'l', 'ow']). """ ans = [] found_empty_prons = False found_large_pronprobs = False # See the comment near the top of this file, RE why we use latin-1. with open(filename, 'r') as f: whitespace = re.compile("[ \t]+") for line in f: a = whitespace.split(line.strip(" \t\r\n")) if len(a) < 2: print("{0}: error: found bad line '{1}' in lexicon file {2} ".format( sys.argv[0], line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) word = a[0] if word == "": # This would clash with the epsilon symbol normally used in OpenFst. print("{0}: error: found as a word in lexicon file " "{1}".format(line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) try: pron_prob = float(a[1]) except: print("{0}: error: found bad line '{1}' in lexicon file {2}, 2nd field " "should be pron-prob".format(sys.argv[0], line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) prons = a[2:] if pron_prob <= 0.0: print("{0}: error: invalid pron-prob in line '{1}' of lexicon file {1} ".format( sys.argv[0], line.strip(" \t\r\n"), filename), file=sys.stderr) sys.exit(1) if len(prons) == 0: found_empty_prons = True ans.append( (word, pron_prob, prons) ) if pron_prob > 1.0: found_large_pronprobs = True if found_empty_prons: print("{0}: warning: found at least one word with an empty pronunciation " "in lexicon file {1}.".format(sys.argv[0], filename), file=sys.stderr) if found_large_pronprobs: print("{0}: warning: found at least one word with pron-prob >1.0 " "in {1}".format(sys.argv[0], filename), file=sys.stderr) if len(ans) == 0: print("{0}: error: found no pronunciations in lexicon file {1}".format( sys.argv[0], filename), file=sys.stderr) sys.exit(1) return ans def write_nonterminal_arcs(start_state, loop_state, next_state, nonterminals, left_context_phones): """This function relates to the grammar-decoding setup, see kaldi-asr.org/doc/grammar.html. It is called from write_fst_no_silence and write_fst_silence, and writes to the stdout some extra arcs in the lexicon FST that relate to nonterminal symbols. See the section "Special symbols in L.fst, kaldi-asr.org/doc/grammar.html#grammar_special_l. start_state: the start-state of L.fst. loop_state: the state of high out-degree in L.fst where words leave and enter. next_state: the number from which this function can start allocating its own states. the updated value of next_state will be returned. nonterminals: the user-defined nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', ... ]. left_context_phones: a list of phones that may appear as left-context, e.g. ['a', 'ah', ... '#nonterm_bos']. """ shared_state = next_state next_state += 1 final_state = next_state next_state += 1 print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=start_state, dest=shared_state, phone='#nonterm_begin', word='#nonterm_begin', cost=0.0)) for nonterminal in nonterminals: print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=loop_state, dest=shared_state, phone=nonterminal, word=nonterminal, cost=0.0)) # this_cost equals log(len(left_context_phones)) but the expression below # better captures the meaning. Applying this cost to arcs keeps the FST # stochatic (sum-to-one, like an HMM), so that if we do weight pushing # things won't get weird. In the grammar-FST code when we splice things # together we will cancel out this cost, see the function CombineArcs(). this_cost = -math.log(1.0 / len(left_context_phones)) for left_context_phone in left_context_phones: print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=shared_state, dest=loop_state, phone=left_context_phone, word='', cost=this_cost)) # arc from loop-state to a final-state with #nonterm_end as ilabel and olabel print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=loop_state, dest=final_state, phone='#nonterm_end', word='#nonterm_end', cost=0.0)) print("{state}\t{final_cost}".format( state=final_state, final_cost=0.0)) return next_state def write_fst_no_silence(lexicon, nonterminals=None, left_context_phones=None): """Writes the text format of L.fst to the standard output. This version is for when --sil-prob=0.0, meaning there is no optional silence allowed. 'lexicon' is a list of 3-tuples (word, pron-prob, prons) as returned by read_lexiconp(). 'nonterminals', which relates to grammar decoding (see kaldi-asr.org/doc/grammar.html), is either None, or the user-defined nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', ... ]. 'left_context_phones', which also relates to grammar decoding, and must be supplied if 'nonterminals' is supplied is either None or a list of phones that may appear as left-context, e.g. ['a', 'ah', ... '#nonterm_bos']. """ loop_state = 0 next_state = 1 # the next un-allocated state, will be incremented as we go. for (word, pronprob, pron) in lexicon: cost = -math.log(pronprob) cur_state = loop_state for i in range(len(pron) - 1): print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=next_state, phone=pron[i], word=(word if i == 0 else ''), cost=(cost if i == 0 else 0.0))) cur_state = next_state next_state += 1 i = len(pron) - 1 # note: i == -1 if pron is empty. print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=loop_state, phone=(pron[i] if i >= 0 else ''), word=(word if i <= 0 else ''), cost=(cost if i <= 0 else 0.0))) if nonterminals is not None: next_state = write_nonterminal_arcs( start_state, loop_state, next_state, nonterminals, left_context_phones) print("{state}\t{final_cost}".format( state=loop_state, final_cost=0.0)) def write_fst_with_silence(lexicon, sil_prob, sil_phone, sil_disambig, nonterminals=None, left_context_phones=None): """Writes the text format of L.fst to the standard output. This version is for when --sil-prob != 0.0, meaning there is optional silence 'lexicon' is a list of 3-tuples (word, pron-prob, prons) as returned by read_lexiconp(). 'sil_prob', which is expected to be strictly between 0.. and 1.0, is the probability of silence 'sil_phone' is the silence phone, e.g. "SIL". 'sil_disambig' is either None, or the silence disambiguation symbol, e.g. "#5". 'nonterminals', which relates to grammar decoding (see kaldi-asr.org/doc/grammar.html), is either None, or the user-defined nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', ... ]. 'left_context_phones', which also relates to grammar decoding, and must be supplied if 'nonterminals' is supplied is either None or a list of phones that may appear as left-context, e.g. ['a', 'ah', ... '#nonterm_bos']. """ assert sil_prob > 0.0 and sil_prob < 1.0 sil_cost = -math.log(sil_prob) no_sil_cost = -math.log(1.0 - sil_prob); start_state = 0 loop_state = 1 # words enter and leave from here sil_state = 2 # words terminate here when followed by silence; this state # has a silence transition to loop_state. next_state = 3 # the next un-allocated state, will be incremented as we go. print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=start_state, dest=loop_state, phone='', word='', cost=no_sil_cost)) print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=start_state, dest=sil_state, phone='', word='', cost=sil_cost)) if sil_disambig is None: print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=sil_state, dest=loop_state, phone=sil_phone, word='', cost=0.0)) else: sil_disambig_state = next_state next_state += 1 print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=sil_state, dest=sil_disambig_state, phone=sil_phone, word='', cost=0.0)) print('{src}\t{dest}\t{phone}\t{word}\t{cost}'.format( src=sil_disambig_state, dest=loop_state, phone=sil_disambig, word='', cost=0.0)) for (word, pronprob, pron) in lexicon: pron_cost = -math.log(pronprob) cur_state = loop_state for i in range(len(pron) - 1): print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=next_state, phone=pron[i], word=(word if i == 0 else ''), cost=(pron_cost if i == 0 else 0.0))) cur_state = next_state next_state += 1 i = len(pron) - 1 # note: i == -1 if pron is empty. print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=loop_state, phone=(pron[i] if i >= 0 else ''), word=(word if i <= 0 else ''), cost=no_sil_cost + (pron_cost if i <= 0 else 0.0))) print("{src}\t{dest}\t{phone}\t{word}\t{cost}".format( src=cur_state, dest=sil_state, phone=(pron[i] if i >= 0 else ''), word=(word if i <= 0 else ''), cost=sil_cost + (pron_cost if i <= 0 else 0.0))) if nonterminals is not None: next_state = write_nonterminal_arcs( start_state, loop_state, next_state, nonterminals, left_context_phones) print("{state}\t{final_cost}".format( state=loop_state, final_cost=0.0)) def write_words_txt(orig_lines, highest_numbered_symbol, nonterminals, filename): """Writes updated words.txt to 'filename'. 'orig_lines' is the original lines in the words.txt file as a list of strings (without the newlines); highest_numbered_symbol is the highest numbered symbol in the original words.txt; nonterminals is a list of strings like '#nonterm:foo'.""" with open(filename, 'w') as f: for l in orig_lines: print(l, file=f) cur_symbol = highest_numbered_symbol + 1 for n in [ '#nonterm_begin', '#nonterm_end' ] + nonterminals: print("{0} {1}".format(n, cur_symbol), file=f) cur_symbol = cur_symbol + 1 def read_nonterminals(filename): """Reads the user-defined nonterminal symbols in 'filename', checks that it has the expected format and has no duplicates, and returns the nonterminal symbols as a list of strings, e.g. ['#nonterm:contact_list', '#nonterm:phone_number', ... ]. """ ans = [line.strip(" \t\r\n") for line in open(filename, 'r')] if len(ans) == 0: raise RuntimeError("The file {0} contains no nonterminals symbols.".format(filename)) for nonterm in ans: if nonterm[:9] != '#nonterm:': raise RuntimeError("In file '{0}', expected nonterminal symbols to start with '#nonterm:', found '{1}'" .format(filename, nonterm)) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def read_left_context_phones(filename): """Reads, checks, and returns a list of left-context phones, in text form, one per line. Returns a list of strings, e.g. ['a', 'ah', ..., '#nonterm_bos' ]""" ans = [line.strip(" \t\r\n") for line in open(filename, 'r')] if len(ans) == 0: raise RuntimeError("The file {0} contains no left-context phones.".format(filename)) whitespace = re.compile("[ \t]+") for s in ans: if len(whitespace.split(s)) != 1: raise RuntimeError("The file {0} contains an invalid line '{1}'".format(filename, s) ) if len(set(ans)) != len(ans): raise RuntimeError("Duplicate nonterminal symbols are present in file {0}".format(filename)) return ans def is_token(s): """Returns true if s is a string and is space-free.""" if not isinstance(s, str): return False whitespace = re.compile("[ \t\r\n]+") split_str = whitespace.split(s); return len(split_str) == 1 and s == split_str[0] def main(args=None): if args is None: args = get_args() lexicon = read_lexiconp(args.lexiconp) if args.nonterminals is None: nonterminals, left_context_phones = None, None else: if args.left_context_phones is None: print("{0}: if --nonterminals is specified, --left-context-phones must also " "be specified".format(sys.argv[0])) sys.exit(1) nonterminals = read_nonterminals(args.nonterminals) left_context_phones = read_left_context_phones(args.left_context_phones) if args.sil_prob == 0.0: write_fst_no_silence(lexicon, nonterminals=nonterminals, left_context_phones=left_context_phones) else: # Do some checking that the options make sense. if args.sil_prob < 0.0 or args.sil_prob >= 1.0: print("{0}: invalid value specified --sil-prob={1}".format( sys.argv[0], args.sil_prob), file=sys.stderr) sys.exit(1) if not is_token(args.sil_phone): print("{0}: you specified --sil-prob={1} but --sil-phone is set " "to '{2}'".format(sys.argv[0], args.sil_prob, args.sil_phone), file=sys.stderr) sys.exit(1) if args.sil_disambig is not None and not is_token(args.sil_disambig): print("{0}: invalid value --sil-disambig='{1}' was specified." "".format(sys.argv[0], args.sil_disambig), file=sys.stderr) sys.exit(1) write_fst_with_silence(lexicon, args.sil_prob, args.sil_phone, args.sil_disambig, nonterminals=nonterminals, left_context_phones=left_context_phones) # (lines, highest_symbol) = read_words_txt(args.input_words_txt) # nonterminals = read_nonterminals(args.nonterminal_symbols_list) # write_words_txt(lines, highest_symbol, nonterminals, args.output_words_txt) if __name__ == '__main__': main() ================================================ FILE: kaldi_active_grammar/model.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # import os, re, shutil from io import open from six import PY2, text_type from . import _log, KaldiError, REQUIRED_MODEL_VERSION from .wfst import SymbolTable from .wrapper import KaldiModelBuildUtils from .utils import ExternalProcess, find_file, load_symbol_table, show_donation_message, symbol_table_lookup import kaldi_active_grammar.defaults as defaults import kaldi_active_grammar.utils as utils _log = _log.getChild('model') ######################################################################################################################## class Lexicon(object): def __init__(self, phones): """ phones: list of strings, each being a phone """ self.phone_set = set(self.make_position_independent(phones)) # XSAMPA phones are 1-letter each, so 2-letter below represent 2 separate phones. CMU_to_XSAMPA_dict = { "'" : "'", 'AA' : 'A', 'AE' : '{', 'AH' : 'V', ## 'AO' : 'O', ## 'AW' : 'aU', 'AY' : 'aI', 'B' : 'b', 'CH' : 'tS', 'D' : 'd', 'DH' : 'D', 'EH' : 'E', 'ER' : '3', 'EY' : 'eI', 'F' : 'f', 'G' : 'g', 'HH' : 'h', 'IH' : 'I', 'IY' : 'i', 'JH' : 'dZ', 'K' : 'k', 'L' : 'l', 'M' : 'm', 'NG' : 'N', 'N' : 'n', 'OW' : 'oU', 'OY' : 'OI', ## 'P' : 'p', 'R' : 'r', 'SH' : 'S', 'S' : 's', 'TH' : 'T', 'T' : 't', 'UH' : 'U', 'UW' : 'u', 'V' : 'v', 'W' : 'w', 'Y' : 'j', 'ZH' : 'Z', 'Z' : 'z', } CMU_to_XSAMPA_dict.update({'AX': '@'}) del CMU_to_XSAMPA_dict["'"] XSAMPA_to_CMU_dict = { v: k for k,v in CMU_to_XSAMPA_dict.items() } # FIXME: handle double-entries @classmethod def phones_cmu_to_xsampa_generic(cls, phones, lexicon_phones=None): new_phones = [] for phone in phones: stress = False if phone.endswith('1'): phone = phone[:-1] stress = True elif phone.endswith(('0', '2')): phone = phone[:-1] phone = cls.CMU_to_XSAMPA_dict[phone] assert 1 <= len(phone) <= 2 new_phone = ("'" if stress else '') + phone if (lexicon_phones is not None) and (new_phone in lexicon_phones): # Add entire possibly-2-letter phone new_phones.append(new_phone) else: # Add each individual 1-letter phone for match in re.finditer(r"('?).", new_phone): new_phones.append(match.group(0)) return new_phones def phones_cmu_to_xsampa(self, phones): return self.phones_cmu_to_xsampa_generic(phones, self.phone_set) @classmethod def make_position_dependent(cls, phones): if len(phones) == 0: return [] elif len(phones) == 1: return [phones[0]+'_S'] else: return [phones[0]+'_B'] + [phone+'_I' for phone in phones[1:-1]] + [phones[-1]+'_E'] @classmethod def make_position_independent(cls, phones): return [re.sub(r'_[SBIE]', '', phone) for phone in phones] @classmethod def generate_pronunciations_cmu_online(cls, word): try: import requests files = {'wordfile': ('wordfile', word)} req = requests.post('http://www.speech.cs.cmu.edu/cgi-bin/tools/logios/lextool.pl', files=files) req.raise_for_status() # FIXME: handle network failures match = re.search(r'', req.text) if match: url = match.group(1) req = requests.get(url) req.raise_for_status() entries = req.text.strip().split('\n') pronunciations = [] for entry in entries: tokens = entry.strip().split() assert re.match(word + r'(\(\d\))?', tokens[0], re.I) # 'SEMI-COLON' or 'SEMI-COLON(2)' phones = tokens[1:] _log.debug("generated pronunciation with cloud-cmudict for %r: CMU phones are %r" % (word, phones)) pronunciations.append(phones) return pronunciations raise KaldiError("received bad response from www.speech.cs.cmu.edu: %r" % req.text) except Exception as e: _log.exception("generate_pronunciations exception accessing www.speech.cs.cmu.edu") raise e g2p_en = None @classmethod def attempt_load_g2p_en(cls, model_dir=None): try: if model_dir: import nltk nltk.data.path.insert(0, os.path.abspath(os.path.join(model_dir, 'g2p'))) # g2p_en>=2.1.0 import g2p_en cls.g2p_en = g2p_en.G2p() assert all(re.sub(r'[012]$', '', phone) in cls.CMU_to_XSAMPA_dict for phone in cls.g2p_en.phonemes if not phone.startswith('<')) except Exception: # including ImportError cls.g2p_en = False # Don't try anymore. _log.debug("failed to load g2p_en") @classmethod def generate_pronunciations_g2p_en(cls, word): try: phones = cls.g2p_en(word) _log.debug("generated pronunciation with g2p_en for %r: %r" % (word, phones)) return [phones] except Exception as e: _log.exception("generate_pronunciations exception using g2p_en") raise e @classmethod def generate_pronunciations(cls, word, model_dir=None, allow_online_pronunciations=False): """returns CMU/arpabet phones""" if cls.g2p_en is None: cls.attempt_load_g2p_en(model_dir) if cls.g2p_en: return cls.generate_pronunciations_g2p_en(word) if allow_online_pronunciations: return cls.generate_pronunciations_cmu_online(word) raise KaldiError("cannot generate word pronunciation: no generators available") ######################################################################################################################## class Model(object): def __init__(self, model_dir=None, tmp_dir=None, tmp_dir_needed=False): show_donation_message() self.model_dir = os.path.join(model_dir or defaults.DEFAULT_MODEL_DIR, '') self.tmp_dir = None if tmp_dir_needed: self.tmp_dir = os.path.join(tmp_dir or os.path.join(self.model_dir, 'cache.tmp'), '') self.exec_dir = os.path.join(utils.exec_dir, '') if not os.path.isdir(self.model_dir): raise KaldiError("cannot find model_dir: %r" % self.model_dir) if self.tmp_dir: if os.path.isfile(self.tmp_dir): raise KaldiError("please specify an available tmp_dir, or remove %r" % self.tmp_dir) if not os.path.exists(self.tmp_dir): _log.warning("%s: creating tmp dir: %r" % (self, self.tmp_dir)) os.mkdir(self.tmp_dir) utils.touch_file(os.path.join(self.tmp_dir, "FILES_ARE_SAFE_TO_DELETE")) if not os.path.isdir(self.exec_dir): raise KaldiError("cannot find exec_dir: %r" % self.exec_dir, "are you sure you installed kaldi-active-grammar correctly?") version_file = os.path.join(self.model_dir, 'KAG_VERSION') if os.path.isfile(version_file): with open(version_file, 'r', encoding='utf-8') as f: model_version = f.read().strip() if model_version != REQUIRED_MODEL_VERSION: raise KaldiError("invalid model_dir version! please download a compatible model") else: _log.warning("model_dir has no version information; errors below may indicate an incompatible model") self.create_missing_files() self.check_user_lexicon() self.files_dict = { 'exec_dir': self.exec_dir, 'model_dir': self.model_dir, 'tmp_dir': self.tmp_dir, 'words.txt': find_file(self.model_dir, 'words.txt', default=True), 'words.base.txt': find_file(self.model_dir, 'words.base.txt', default=True), 'phones.txt': find_file(self.model_dir, 'phones.txt', default=True), 'align_lexicon.int': find_file(self.model_dir, 'align_lexicon.int', default=True), 'align_lexicon.base.int': find_file(self.model_dir, 'align_lexicon.base.int', default=True), 'disambig.int': find_file(self.model_dir, 'disambig.int', default=True), 'L_disambig.fst': find_file(self.model_dir, 'L_disambig.fst', default=True), 'tree': find_file(self.model_dir, 'tree', default=True), 'final.mdl': find_file(self.model_dir, 'final.mdl', default=True), # 'g.irelabel': find_file(self.model_dir, 'g.irelabel', default=True), # otf 'user_lexicon.txt': find_file(self.model_dir, 'user_lexicon.txt', default=True), 'left_context_phones.txt': find_file(self.model_dir, 'left_context_phones.txt', default=True), 'nonterminals.txt': find_file(self.model_dir, 'nonterminals.txt', default=True), 'wdisambig_phones.int': find_file(self.model_dir, 'wdisambig_phones.int', default=True), 'wdisambig_words.int': find_file(self.model_dir, 'wdisambig_words.int', default=True), 'lexiconp_disambig.txt': find_file(self.model_dir, 'lexiconp_disambig.txt', default=True), 'lexiconp_disambig.base.txt': find_file(self.model_dir, 'lexiconp_disambig.base.txt', default=True), 'words.relabeled.txt': find_file(self.model_dir, 'words.relabeled.txt', default=True), } self.files_dict.update({ k.replace('.', '_'): v for (k, v) in self.files_dict.items() }) # For named placeholder access in str.format() self.fst_cache = utils.FSTFileCache(os.path.join(self.model_dir, defaults.FILE_CACHE_FILENAME), dependencies_dict=self.files_dict, tmp_dir=self.tmp_dir) self.phone_to_int_dict = { phone: i for phone, i in load_symbol_table(self.files_dict['phones.txt']) } self.lexicon = Lexicon(self.phone_to_int_dict.keys()) self.nonterm_phones_offset = self.phone_to_int_dict.get('#nonterm_bos') if self.nonterm_phones_offset is None: raise KaldiError("missing nonterms in 'phones.txt'") self.nonterm_words_offset = symbol_table_lookup(self.files_dict['words.base.txt'], '#nonterm_begin') if self.nonterm_words_offset is None: raise KaldiError("missing nonterms in 'words.base.txt'") # Update files if needed, before loading words necessary_files = ['user_lexicon.txt', 'words.txt',] non_lazy_files = ['align_lexicon.int', 'lexiconp_disambig.txt', 'L_disambig.fst',] files_are_not_current = lambda files: any(not self.fst_cache.file_is_current(self.files_dict[file]) for file in files) if self.fst_cache.cache_is_new or files_are_not_current(necessary_files + non_lazy_files): self.generate_lexicon_files() self.words_table = SymbolTable() self.load_words() def load_words(self, words_file=None): if words_file is None: words_file = self.files_dict['words.txt'] _log.debug("loading words from %r", words_file) invalid_words = " !SIL #0 ".lower().split() self.words_table.load_text_file(words_file) self.longest_word = max(self.words_table.word_to_id_map.keys(), key=len) return self.words_table def read_user_lexicon(self, filename=None): if filename is None: filename = self.files_dict['user_lexicon.txt'] with open(filename, 'r', encoding='utf-8') as file: entries = [line.split() for line in file if line.split()] for tokens in entries: # word lowercase tokens[0] = tokens[0].lower() return entries def write_user_lexicon(self, entries, filename=None): if filename is None: filename = self.files_dict['user_lexicon.txt'] lines = [' '.join(tokens) + '\n' for tokens in entries] with open(filename, 'w', encoding='utf-8', newline='\n') as file: file.writelines(lines) def add_word(self, word, phones=None, lazy_compilation=False, allow_online_pronunciations=False): word = word.strip().lower() if phones is None: # Not given pronunciation(s), so generate pronunciation(s), then call ourselves recursively for each individual pronunciation pronunciations = Lexicon.generate_pronunciations(word, model_dir=self.model_dir, allow_online_pronunciations=allow_online_pronunciations) pronunciations = sum([ self.add_word(word, phones, lazy_compilation=True) for phones in pronunciations], []) if not lazy_compilation: self.generate_lexicon_files() return pronunciations # FIXME: refactor this function # Now just handle single-pronunciation case... phones = self.lexicon.phones_cmu_to_xsampa(phones) new_entry = [word] + phones entries = self.read_user_lexicon() if any(new_entry == entry for entry in entries): _log.warning("word & pronunciation already in user_lexicon") return [phones] for tokens in entries: if word == tokens[0]: _log.warning("word (with different pronunciation) already in user_lexicon: %s" % tokens[1:]) entries.append(new_entry) self.write_user_lexicon(entries) if lazy_compilation: self.words_table.add_word(word) else: self.generate_lexicon_files() return [phones] def create_missing_files(self): utils.touch_file(os.path.join(self.model_dir, 'user_lexicon.txt')) def check_file(filename, src_filename): # Create missing file from its base file if not find_file(self.model_dir, filename): src = find_file(self.model_dir, src_filename) dst = src.replace(src_filename, filename) shutil.copyfile(src, dst) check_file('words.txt', 'words.base.txt') check_file('align_lexicon.int', 'align_lexicon.base.int') check_file('lexiconp_disambig.txt', 'lexiconp_disambig.base.txt') def check_user_lexicon(self): """ Checks for a user lexicon file in the CWD, and if found and different than the model's user lexicon, extends the model's. """ cwd_user_lexicon_filename = os.path.abspath('user_lexicon.txt') model_user_lexicon_filename = os.path.abspath(os.path.join(self.model_dir, 'user_lexicon.txt')) if (cwd_user_lexicon_filename != model_user_lexicon_filename) and os.path.isfile(cwd_user_lexicon_filename): cwd_user_lexicon_entries = [tuple(tokens) for tokens in self.read_user_lexicon(filename=cwd_user_lexicon_filename)] model_user_lexicon_entries = [tuple(tokens) for tokens in self.read_user_lexicon(filename=model_user_lexicon_filename)] model_user_lexicon_entries_set = set(model_user_lexicon_entries) new_user_lexicon_entries = [tokens for tokens in cwd_user_lexicon_entries if tokens not in model_user_lexicon_entries_set] if new_user_lexicon_entries: _log.info("adding new user lexicon entries from %r", cwd_user_lexicon_filename) entries = model_user_lexicon_entries + new_user_lexicon_entries self.write_user_lexicon(entries, filename=model_user_lexicon_filename) def generate_lexicon_files(self): """ Generates: words.txt, align_lexicon.int, lexiconp_disambig.txt, L_disambig.fst """ _log.info("generating lexicon files") self.fst_cache.invalidate() # FIXME: refactor this to use words_table/SymbolTable max_word_id = max(word_id for word, word_id in load_symbol_table(base_filepath(self.files_dict['words.txt'])) if word_id < self.nonterm_words_offset) user_lexicon_entries = [] with open(self.files_dict['user_lexicon.txt'], 'r', encoding='utf-8') as user_lexicon: for line in user_lexicon: tokens = line.split() if len(tokens) >= 2: word, phones = tokens[0], tokens[1:] phones = Lexicon.make_position_dependent(phones) unknown_phones = [phone for phone in phones if phone not in self.phone_to_int_dict] if unknown_phones: raise KaldiError("word %r has unknown phone(s) %r" % (word, unknown_phones)) # _log.critical("word %r has unknown phone(s) %r so using junk phones!!!", word, unknown_phones) # phones = [phone if phone not in self.phone_to_int_dict else self.noise_phone for phone in phones] # continue max_word_id += 1 user_lexicon_entries.append((word, max_word_id, phones)) def generate_file_from_base_with_user_lexicon(filename, write_func): filepath = self.files_dict[filename] with open(base_filepath(filepath), 'r', encoding='utf-8') as file: base_data = file.read() with open(filepath, 'w', encoding='utf-8', newline='\n') as file: file.write(base_data) for word, word_id, phones in user_lexicon_entries: file.write(write_func(word, word_id, phones) + '\n') generate_file_from_base_with_user_lexicon('words.txt', lambda word, word_id, phones: str_space_join([word, word_id])) generate_file_from_base_with_user_lexicon('align_lexicon.int', lambda word, word_id, phones: str_space_join([word_id, word_id] + [self.phone_to_int_dict[phone] for phone in phones])) generate_file_from_base_with_user_lexicon('lexiconp_disambig.txt', lambda word, word_id, phones: '%s\t1.0 %s' % (word, ' '.join(phones))) if True: lexicon_fst_text = KaldiModelBuildUtils.make_lexicon_fst( left_context_phones=self.files_dict['left_context_phones_txt'], nonterminals=self.files_dict['nonterminals_txt'], sil_prob=0.5, sil_phone='SIL', sil_disambig='#14', # FIXME: lookup correct value lexiconp=self.files_dict['lexiconp_disambig_txt'], ) KaldiModelBuildUtils.build_L_disambig( lexicon_fst_text.encode(encoding='latin-1'), phones_file=self.files_dict['phones_txt'], words_file=self.files_dict['words_txt'], wdisambig_phones_file=self.files_dict['wdisambig_phones_int'], wdisambig_words_file=self.files_dict['wdisambig_words_int'], fst_out_file=self.files_dict['L_disambig_fst']) else: format = ExternalProcess.get_list_formatter(self.files_dict) command = ExternalProcess.make_lexicon_fst(*format( '--left-context-phones={left_context_phones_txt}', '--nonterminals={nonterminals_txt}', '--sil-prob=0.5', '--sil-phone=SIL', '--sil-disambig=#14', # FIXME: lookup correct value '{lexiconp_disambig_txt}', )) command |= ExternalProcess.fstcompile(*format( '--isymbols={phones_txt}', '--osymbols={words_txt}', '--keep_isymbols=false', '--keep_osymbols=false', )) command |= ExternalProcess.fstaddselfloops(*format('{wdisambig_phones_int}', '{wdisambig_words_int}'), **ExternalProcess.get_debug_stderr_kwargs(_log)) command |= ExternalProcess.fstarcsort(*format('--sort_type=olabel')) command |= self.files_dict['L_disambig.fst'] command() # FIXME: generate_words_relabeled_file(self.files_dict['words.txt'], self.files_dict['relabel_ilabels.int'], self.files_dict['words.relabeled.txt']) self.fst_cache.update_dependencies() self.fst_cache.save() def reset_user_lexicon(self): utils.clear_file(self.files_dict['user_lexicon.txt']) self.generate_lexicon_files() @staticmethod def generate_words_relabeled_file(words_filename, relabel_filename, words_relabel_filename): """ generate a version of the words file, that has already been relabeled with the given relabel file """ with open(words_filename, 'r', encoding='utf-8') as file: word_id_pairs = [(word, id) for (word, id) in [line.strip().split() for line in file]] with open(relabel_filename, 'r', encoding='utf-8') as file: relabel_map = {from_id: to_id for (from_id, to_id) in [line.strip().split() for line in file]} word_ids = frozenset(id for (word, id) in word_id_pairs) relabel_from_ids = frozenset(from_id for from_id in relabel_map.keys()) if word_ids < relabel_from_ids: _log.warning("generate_words_relabeled_file: word_ids < relabel_from_ids") # if word_ids > relabel_from_ids: # _log.warning("generate_words_relabeled_file: word_ids > relabel_from_ids") with open(words_relabel_filename, 'w', encoding='utf-8') as file: for (word, id) in word_id_pairs: file.write("%s %s\n" % (word, (relabel_map.get(id, id)))) ######################################################################################################################## def convert_generic_model_to_agf(src_dir, model_dir): from .compiler import Compiler if PY2: from .kaldi import augment_phones_txt_py2 as augment_phones_txt, augment_words_txt_py2 as augment_words_txt else: from .kaldi import augment_phones_txt, augment_words_txt filenames = [ 'words.txt', 'phones.txt', 'align_lexicon.int', 'disambig.int', # 'L_disambig.fst', 'tree', 'final.mdl', 'lexiconp.txt', 'word_boundary.txt', 'optional_silence.txt', 'silence.txt', 'nonsilence.txt', 'wdisambig_phones.int', 'wdisambig_words.int', 'mfcc_hires.conf', 'mfcc.conf', 'ivector_extractor.conf', 'splice.conf', 'online_cmvn.conf', 'final.mat', 'global_cmvn.stats', 'final.dubm', 'final.ie', ] nonterminals = list(Compiler.nonterminals) for filename in filenames: path = find_file(src_dir, filename) if path is None: _log.error("cannot find %r in %r", filename, model_dir) continue _log.info("copying %r to %r", path, model_dir) shutil.copy(path, model_dir) _log.info("converting %r in %r", 'phones.txt', model_dir) lines, highest_symbol = augment_phones_txt.read_phones_txt(os.path.join(model_dir, 'phones.txt')) augment_phones_txt.write_phones_txt(lines, highest_symbol, nonterminals, os.path.join(model_dir, 'phones.txt')) _log.info("converting %r in %r", 'words.txt', model_dir) lines, highest_symbol = augment_words_txt.read_words_txt(os.path.join(model_dir, 'words.txt')) # FIXME: leave space for adding words later augment_words_txt.write_words_txt(lines, highest_symbol, nonterminals, os.path.join(model_dir, 'words.txt')) with open(os.path.join(model_dir, 'nonterminals.txt'), 'w', encoding='utf-8', newline='\n') as f: f.writelines(nonterm + '\n' for nonterm in nonterminals) # add nonterminals to align_lexicon.int # fix L_disambig.fst: construct lexiconp_disambig.txt ... ######################################################################################################################## def str_space_join(iterable): return u' '.join(text_type(elem) for elem in iterable) def base_filepath(filepath): root, ext = os.path.splitext(filepath) return root + '.base' + ext def verify_files_exist(*filenames): return False ================================================ FILE: kaldi_active_grammar/plain_dictation.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # from . import _log, KaldiError from .model import Model from .compiler import Compiler, remove_nonterms_in_text, remove_words_in_text from .wrapper import KaldiPlainNNet3Decoder, KaldiAgfNNet3Decoder from .utils import show_donation_message _log = _log.getChild('plain_dictation') class PlainDictationRecognizer(object): def __init__(self, model_dir=None, tmp_dir=None, fst_file=None, config=None): """ Recognizes plain dictation only. If `fst_file` is specified, uses that HCLG.fst file; otherwise, uses KaldiAG but dictation only. Args: model_dir (str): optional path to model directory tmp_dir (str): optional path to temporary directory fst_file (str): optional path to model's HCLG.fst file to use config (dict): optional configuration for initialization of decoder """ show_donation_message() kwargs = {} if config: kwargs['config'] = dict(config) if fst_file: self._model = Model(model_dir, tmp_dir) self.decoder = KaldiPlainNNet3Decoder(model_dir=self._model.model_dir, tmp_dir=self._model.tmp_dir, fst_file=fst_file, **kwargs) else: self._compiler = Compiler(model_dir, tmp_dir, cache_fsts=False) top_fst_rule = self._compiler.compile_top_fst_dictation_only() dictation_fst_file = self._compiler.dictation_fst_filepath self.decoder = KaldiAgfNNet3Decoder(model_dir=self._compiler.model_dir, tmp_dir=self._compiler.tmp_dir, top_fst=top_fst_rule.fst_wrapper, dictation_fst_file=dictation_fst_file, **kwargs) def decode_utterance(self, samples_data, chunk_size=None): """ Decodes an entire utterance at once, taking as input *samples_data* (*bytes-like* in `int16` format), and returning a tuple of (output (*text*), likelihood (*float*)). Optionally takes *chunk_size* (*int* in number of samples) for decoding. """ if chunk_size: chunk_size *= 2 # Compensate for int16 format for i in range(0, len(samples_data), chunk_size): self.decoder.decode(samples_data[i : i + chunk_size], False) self.decoder.decode(bytes(), True) else: self.decoder.decode(samples_data, True) output_str, info = self.decoder.get_output() output_str = remove_nonterms_in_text(output_str) output_str = remove_words_in_text(output_str, lambda word: word in self._compiler._silence_words) return (output_str, info) ================================================ FILE: kaldi_active_grammar/utils.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # import logging, sys, time import fnmatch, glob, os import functools import hashlib, json import threading from contextlib import contextmanager from io import open import six from six import PY2, binary_type, text_type, print_ from . import _log, _name, __version__ ######################################################################################################################## _donation_message_enabled = True _donation_message = ("Kaldi-Active-Grammar v%s: \n" " If this free, open source engine is valuable to you, please consider donating \n" " https://github.com/daanzu/kaldi-active-grammar \n" " Disable message by calling `kaldi_active_grammar.disable_donation_message()`") % __version__ def show_donation_message(): if _donation_message_enabled: print_(_donation_message) disable_donation_message() def disable_donation_message(): global _donation_message_enabled _donation_message_enabled = False ######################################################################################################################## debug_timer_enabled = True class ThreadLocalData(threading.local): def __init__(self): self._debug_timer_stack = [] thread_local_data = ThreadLocalData() @contextmanager def debug_timer(log, desc, enabled=True, independent=False): """ Contextmanager that outputs timing to ``log`` with ``desc``. :param independent: if True, tracks entire time spent inside context, rather than subtracting time within inner ``debug_timer`` instances """ _debug_timer_stack = thread_local_data._debug_timer_stack start_time = time.time() if not independent: _debug_timer_stack.append(start_time) spent_time_func = lambda: time.time() - start_time yield spent_time_func if not independent: start_time_adjusted = _debug_timer_stack.pop() else: start_time_adjusted = 0 if enabled: if debug_timer_enabled: log("%s %d ms" % (desc, (time.time() - start_time_adjusted) * 1000)) if _debug_timer_stack and not independent: _debug_timer_stack[-1] += spent_time_func() if not PY2: def clock(): return time.perf_counter() else: def clock(): return time.clock() ######################################################################################################################## if sys.platform.startswith('win'): platform = 'windows' elif sys.platform.startswith('linux'): platform = 'linux' elif sys.platform.startswith('darwin'): platform = 'macos' else: raise KaldiError("unknown sys.platform") exec_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'exec', platform) import ush class ExternalProcess(object): shell = ush.Shell(raise_on_error=True) fstcompile = shell(os.path.join(exec_dir, 'fstcompile')) fstarcsort = shell(os.path.join(exec_dir, 'fstarcsort')) fstaddselfloops = shell(os.path.join(exec_dir, 'fstaddselfloops')) fstinfo = shell(os.path.join(exec_dir, 'fstinfo')) # compile_graph = shell(os.path.join(exec_dir, 'compile-graph')) compile_graph_agf = shell(os.path.join(exec_dir, 'compile-graph-agf')) # compile_graph_agf_debug = shell(os.path.join(exec_dir, 'compile-graph-agf-debug')) make_lexicon_fst = shell([sys.executable, os.path.join(os.path.dirname(os.path.abspath(__file__)), 'kaldi', 'make_lexicon_fst%s.py' % ('_py2' if PY2 else ''))]) @staticmethod def get_dict_formatter(format_kwargs): return lambda **kwargs: { key: value.format(**format_kwargs) for (key, value) in kwargs.items() } @staticmethod def get_list_formatter(format_kwargs): return lambda *args: [arg.format(**format_kwargs) for arg in args] @staticmethod def get_debug_stderr_kwargs(log): return (dict() if log.isEnabledFor(logging.DEBUG) else dict(stderr=six.BytesIO())) @staticmethod def execute_command_safely(commands, log): """ Executes given `ush` command, redirecting stderr appropriately: either logging, or storing to output upon error. """ stderr = six.BytesIO() for command in commands.commands: command.opts['stderr'] = stderr try: result = commands() except Exception as e: log.error("Error running command. Printing stderr as follows...\n%s", stderr.getvalue().decode('utf-8')) raise e return result ######################################################################################################################## def lazy_readonly_property(func): # From https://stackoverflow.com/questions/3012421/python-memoising-deferred-lookup-property-decorator attr_name = '_lazy_' + func.__name__ @property @functools.wraps(func) def _lazyprop(self): if not hasattr(self, attr_name): setattr(self, attr_name, func(self)) return getattr(self, attr_name) return _lazyprop class lazy_settable_property(object): ''' meant to be used for lazy evaluation of an object attribute. property should represent non-mutable data, as it replaces itself. ''' # From https://stackoverflow.com/questions/3012421/python-memoising-deferred-lookup-property-decorator def __init__(self, fget): self.fget = fget # copy the getter function's docstring and other attributes functools.update_wrapper(self, fget) def __get__(self, obj, cls): if obj is None: return self value = self.fget(obj) setattr(obj, self.fget.__name__, value) return value ######################################################################################################################## def touch_file(filename): with open(filename, 'ab'): os.utime(filename, None) # Update timestamps def clear_file(filename): with open(filename, 'wb'): pass symbol_table_lookup_cache = dict() def symbol_table_lookup(filename, input): """ Returns the RHS corresponding to LHS == ``input`` in symbol table in ``filename``. """ cached = symbol_table_lookup_cache.get((filename, input)) if cached is not None: return cached with open(filename, 'r', encoding='utf-8') as f: for line in f: tokens = line.strip().split() if len(tokens) >= 2 and input == tokens[0]: try: symbol_table_lookup_cache[(filename, input)] = int(tokens[1]) return int(tokens[1]) except Exception as e: symbol_table_lookup_cache[(filename, input)] = tokens[1] return tokens[1] return None def load_symbol_table(filename): with open(filename, 'r', encoding='utf-8') as f: return [[int(token) if token.isdigit() else token for token in line.strip().split()] for line in f] def find_file(directory, filename, required=False, default=False): matches = [] for root, dirnames, filenames in os.walk(directory): for filename in fnmatch.filter(filenames, filename): matches.append(os.path.join(root, filename)) if matches: matches.sort(key=len) _log.log(8, "%s: find_file found file %r", _name, matches[0]) return matches[0] else: _log.log(8, "%s: find_file cannot find file %r in %r (or subdirectories)", _name, filename, directory) if required: raise IOError("cannot find file %r in %r" % (filename, directory)) if default == True: return os.path.join(directory, filename) return None def is_file_up_to_date(filename, *parent_filenames): if not os.path.exists(filename): return False for parent_filename in parent_filenames: if not os.path.exists(parent_filename): return False if os.path.getmtime(filename) < os.path.getmtime(parent_filename): return False return True ######################################################################################################################## class FSTFileCache(object): def __init__(self, cache_filename, tmp_dir=None, dependencies_dict=None, invalidate=False): """ Stores mapping filename -> hash of its contents/data, to detect when recalculaion is necessary. Assumes file is in model_dir. Also stores an entry ``dependencies_list`` listing filenames of all dependencies. FST files are a special case: they aren't stored in the cache object, because their filename is itself a hash of its content mixed with a hash of its dependencies. If ``invalidate``, then initialize a fresh cache. """ self.cache_filename = cache_filename self.tmp_dir = tmp_dir if dependencies_dict is None: dependencies_dict = dict() self.dependencies_dict = dependencies_dict self.lock = threading.Lock() try: self._load() except Exception as e: _log.info("%s: failed to load cache from %r", self, cache_filename) self.cache = None must_reset_cache = False if invalidate: _log.debug("%s: forced invalidate", self) must_reset_cache = True elif self.cache is None: _log.debug("%s: could not load cache", self) must_reset_cache = True elif self.cache.get('version') != __version__: _log.debug("%s: version changed", self) must_reset_cache = True elif sorted(self.cache.get('dependencies_list', list())) != sorted(dependencies_dict.keys()): _log.debug("%s: list of dependencies has changed", self) must_reset_cache = True elif any(not self.file_is_current(path) for (name, path) in dependencies_dict.items() if path and os.path.isfile(path)): _log.debug("%s: any of the dependencies files' contents (as stored in cache) has changed", self) must_reset_cache = True if must_reset_cache: # Then reset cache _log.info("%s: version or dependencies did not match cache from %r; initializing empty", self, cache_filename) self.cache = dict({ 'version': text_type(__version__) }) self.cache_is_new = True self.update_dependencies() self.save() def _load(self): with open(self.cache_filename, 'r', encoding='utf-8') as f: self.cache = json.load(f) self.cache_is_new = False self.dirty = False dependencies_hash = property(lambda self: self.cache['dependencies_hash']) def save(self): with open(self.cache_filename, 'w', encoding='utf-8') as f: # https://stackoverflow.com/a/14870531 f.write(json.dumps(self.cache, ensure_ascii=False)) self.dirty = False def update_dependencies(self): dependencies_dict = self.dependencies_dict for (name, path) in dependencies_dict.items(): if path and os.path.isfile(path): self.add_file(path) self.cache['dependencies_list'] = sorted(dependencies_dict.keys()) # list self.cache['dependencies_hash'] = self.hash_data([self.cache.get(path) for (key, path) in sorted(dependencies_dict.items())]) def invalidate(self, filename=None): if filename is None: _log.info("%s: invalidating all file entries in cache", self) # Does not invalidate dependencies! self.cache = { key: self.cache[key] for key in ['version', 'dependencies_list', 'dependencies_hash'] + self.cache['dependencies_list'] if key in self.cache } self.dirty = True if self.tmp_dir is not None: for filename in glob.glob(os.path.join(self.tmp_dir, '*.fst')): os.remove(filename) elif filename in self.cache: _log.info("%s: invalidating cache entry for %r", self, filename) del self.cache[filename] self.dirty = True def hash_data(self, data, mix_dependencies=False): if not isinstance(data, binary_type): if not isinstance(data, text_type): data = text_type(data) data = data.encode('utf-8') hasher = hashlib.md5() if mix_dependencies: hasher.update(self.dependencies_hash.encode('utf-8')) hasher.update(data) return text_type(hasher.hexdigest()) def add_file(self, filepath, data=None): # Assumes file is a root dependency if data is None: with open(filepath, 'rb') as f: data = f.read() filename = os.path.basename(filepath) self.cache[filename] = self.hash_data(data) self.dirty = True def contains(self, filename, data): return (filename in self.cache) and (self.cache[filename] == self.hash_data(data)) def file_is_current(self, filepath, data=None): """Returns bool whether generic filepath file exists and the cache contains the given data (or the file's current data if none given).""" filename = os.path.basename(filepath) if self.cache_is_new and filename in self.cache.get('dependencies_list', list()): return False if not os.path.isfile(filepath): return False if data is None: with open(filepath, 'rb') as f: data = f.read() return self.contains(filename, data) def fst_is_current(self, filepath, touch=True): """Returns bool whether FST file in directory path exists.""" result = os.path.isfile(filepath) if result and touch: touch_file(filepath) return result ================================================ FILE: kaldi_active_grammar/wfst.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # import collections, itertools, math from six import iteritems, itervalues, text_type from . import KaldiError from .utils import FSTFileCache class WFST(object): """ WFST class. Notes: * Weight (arc & state) is stored as raw probability, then normalized and converted to negative log likelihood/probability before export. """ zero = float('inf') # Weight of non-final states; a state is final if and only if its weight is not equal to self.zero one = 0.0 eps = u'' eps_disambig = u'#0' silent_labels = frozenset((eps, eps_disambig, u'!SIL')) native = property(lambda self: False) def __init__(self): self.clear() def clear(self): self._arc_table_dict = collections.defaultdict(list) # { src_state: [[src_state, dst_state, label, olabel, weight], ...] } # list of its outgoing arcs self._state_table = dict() # { id: weight } self._next_state_id = 0 self.start_state = self.add_state() self.filename = None num_arcs = property(lambda self: sum(len(arc_list) for arc_list in itervalues(self._arc_table_dict))) num_states = property(lambda self: len(self._state_table)) def iter_arcs(self): return itertools.chain.from_iterable(itervalues(self._arc_table_dict)) def is_state_final(self, state): return (self._state_table[state] != 0) def add_state(self, weight=None, initial=False, final=False): """ Default weight is 1. """ self.filename = None id = int(self._next_state_id) self._next_state_id += 1 if weight is None: weight = 1 if final else 0 else: assert final self._state_table[id] = float(weight) if initial: self.add_arc(self.start_state, id, None) return id def add_arc(self, src_state, dst_state, label, olabel=None, weight=None): """ Default weight is 1. None label is replaced by eps. Default olabel of None is replaced by label. """ self.filename = None if label is None: label = self.eps if olabel is None: olabel = label if weight is None: weight = 1 self._arc_table_dict[src_state].append( [int(src_state), int(dst_state), text_type(label), text_type(olabel), float(weight)]) def get_fst_text(self, fst_cache, eps2disambig=False): eps_replacement = self.eps_disambig if eps2disambig else self.eps arcs_text = u''.join("%d %d %s %s %f\n" % ( src_state, dst_state, ilabel if ilabel != self.eps else eps_replacement, olabel, -math.log(weight) if weight != 0 else self.zero, ) for (src_state, dst_state, ilabel, olabel, weight) in self.iter_arcs()) states_text = u''.join("%d %f\n" % ( id, -math.log(weight) if weight != 0 else self.zero, ) for (id, weight) in iteritems(self._state_table) if weight != 0) text = arcs_text + states_text self.filename = fst_cache.hash_data(text, mix_dependencies=True) + '.fst' return text #################################################################################################################### def label_is_silent(self, label): return ((label in self.silent_labels) or (label.startswith('#nonterm'))) def scale_weights(self, factor): # Unused factor = float(factor) for arcs in itervalues(self._arc_table_dict): for arc in arcs: arc[4] = arc[4] * factor def normalize_weights(self, stochasticity=False): # Unused for arcs in itervalues(self._arc_table_dict): num_weights = len(arcs) sum_weights = sum(arc[4] for arc in arcs) divisor = float(sum_weights if stochasticity else num_weights) for arc in arcs: arc[4] = arc[4] / divisor def has_eps_path(self, path_src_state, path_dst_state, eps_like_labels=frozenset()): """ Returns True iff there is a epsilon path from src_state to dst_state. Uses BFS. Does not follow nonterminals! Used by Dragonfly compiler. """ eps_like_labels = frozenset((self.eps, self.eps_disambig)) | frozenset(eps_like_labels) state_queue = collections.deque([path_src_state]) queued = set(state_queue) while state_queue: state = state_queue.pop() if state == path_dst_state: return True next_states = [dst_state for (src_state, dst_state, label, olabel, weight) in self._arc_table_dict[state] if (label in eps_like_labels) and (dst_state not in queued)] state_queue.extendleft(next_states) queued.update(next_states) return False def does_match(self, target_words, wildcard_nonterms=(), include_silent=False): """ Returns the olabels on a matching path if there is one, False if not. Uses BFS. Wildcard accepts zero or more words. Used for parsing by KaldiAG.compiler. """ queue = collections.deque() # entries: (state, path of olabels of arcs to state, index into target_words of remaining words) queue.append((self.start_state, (), 0)) while queue: state, path, target_word_index = queue.popleft() target_word = target_words[target_word_index] if target_word_index < len(target_words) else None if (target_word is None) and self.is_state_final(state): return tuple(olabel for olabel in path if include_silent or not self.label_is_silent(olabel)) for arc in self._arc_table_dict[state]: src_state, dst_state, ilabel, olabel, weight = arc if (target_word is not None) and (ilabel == target_word): queue.append((dst_state, path+(olabel,), target_word_index+1)) elif ilabel in wildcard_nonterms: if olabel not in path: path += (olabel,) # FIXME: Is this right? shouldn't we only check for olabel at end of path? if target_word is not None: queue.append((src_state, path+(target_word,), target_word_index+1)) # accept word and stay queue.append((dst_state, path, target_word_index)) # epsilon transition; already added olabel above or previously elif self.label_is_silent(ilabel): queue.append((dst_state, path+(olabel,), target_word_index)) # epsilon transition return False ######################################################################################################################## from .ffi import FFIObject, _ffi, decode, encode class NativeWFST(FFIObject): """ WFST class, implemented in native code. Notes: * Weight (arc & state) is stored as raw probability, then normalized and converted to negative log likelihood/probability before export. """ _library_header_text = """ DRAGONFLY_API bool fst__init(int32_t eps_like_ilabels_len, int32_t eps_like_ilabels_cp[], int32_t silent_olabels_len, int32_t silent_olabels_cp[], int32_t wildcard_olabels_len, int32_t wildcard_olabels_cp[]); DRAGONFLY_API void* fst__construct(); DRAGONFLY_API bool fst__destruct(void* fst_vp); DRAGONFLY_API int32_t fst__add_state(void* fst_vp, float weight, bool initial); DRAGONFLY_API bool fst__add_arc(void* fst_vp, int32_t src_state_id, int32_t dst_state_id, int32_t ilabel, int32_t olabel, float weight); DRAGONFLY_API bool fst__compute_md5(void* fst_vp, char* md5_cp, char* dependencies_seed_md5_cp); DRAGONFLY_API bool fst__has_path(void* fst_vp); DRAGONFLY_API bool fst__has_eps_path(void* fst_vp, int32_t path_src_state, int32_t path_dst_state); DRAGONFLY_API bool fst__does_match(void* fst_vp, int32_t target_labels_len, int32_t target_labels_cp[], int32_t output_labels_cp[], int32_t* output_labels_len); DRAGONFLY_API void* fst__load_file(char* filename_cp); DRAGONFLY_API bool fst__write_file(void* fst_vp, char* filename_cp); DRAGONFLY_API bool fst__write_file_const(void* fst_vp, char* filename_cp); DRAGONFLY_API bool fst__print(void* fst_vp, char* filename_cp); DRAGONFLY_API void* fst__compile_text(char* fst_text_cp, char* isymbols_file_cp, char* osymbols_file_cp); """ zero = float('inf') # Weight of non-final states; a state is final if and only if its weight is not equal to self.zero one = 0.0 eps = u'' eps_disambig = u'#0' silent_words = frozenset((eps, eps_disambig, u'!SIL')) native = property(lambda self: True) @classmethod def init_class(cls, isymbol_table, wildcard_nonterms, osymbol_table=None): if osymbol_table is None: osymbol_table = isymbol_table cls.word_to_ilabel_map = isymbol_table.word_to_id_map cls.word_to_olabel_map = osymbol_table.word_to_id_map cls.olabel_to_word_map = osymbol_table.id_to_word_map cls.eps_like_ilabels = tuple(cls.word_to_ilabel_map[word] for word in (cls.eps, cls.eps_disambig)) cls.silent_olabels = tuple( frozenset(cls.word_to_olabel_map[word] for word in cls.silent_words) | frozenset(symbol for (word, symbol) in cls.word_to_olabel_map.items() if word.startswith('#nonterm'))) cls.wildcard_nonterms = frozenset(wildcard_nonterms) cls.wildcard_olabels = tuple(cls.word_to_olabel_map[word] for word in cls.wildcard_nonterms) assert cls.word_to_ilabel_map[cls.eps] == 0 cls.init_ffi() result = cls._lib.fst__init(len(cls.eps_like_ilabels), cls.eps_like_ilabels, len(cls.silent_olabels), cls.silent_olabels, len(cls.wildcard_olabels), cls.wildcard_olabels) if not result: raise KaldiError("Failed fst__init") def __init__(self): super().__init__() self._construct() def _construct(self): self.native_obj = self._lib.fst__construct() if self.native_obj == _ffi.NULL: raise KaldiError("Failed fst__construct") self.num_states = 1 # Is initialized with a start state self.num_arcs = 0 self.filename = None self._compiled_native_obj = None def __del__(self): self.destruct() def destruct(self): del self.compiled_native_obj if self.native_obj is not None: result = self._lib.fst__destruct(self.native_obj) self.native_obj = None if not result: raise KaldiError("Failed fst__destruct on %r" % self.native_obj) compiled_native_obj = property(lambda self: self._compiled_native_obj) @compiled_native_obj.setter def compiled_native_obj(self, value): del self.compiled_native_obj self._compiled_native_obj = value @compiled_native_obj.deleter def compiled_native_obj(self): if self._compiled_native_obj is not None: result = self._lib.fst__destruct(self._compiled_native_obj) self._compiled_native_obj = None if not result: raise KaldiError("Failed fst__destruct on %r" % self._compiled_native_obj) def clear(self): self.destruct() self._construct() def add_state(self, weight=None, initial=False, final=False): """ Default weight is 1. """ self.filename = None if weight is None: weight = 1 if final else 0 else: assert final weight = -math.log(weight) if weight != 0 else self.zero id = self._lib.fst__add_state(self.native_obj, float(weight), bool(initial)) if id < 0: raise KaldiError("Failed fst__add_state") self.num_states += 1 if initial: self.num_arcs += 1 return id def add_arc(self, src_state, dst_state, label, olabel=None, weight=None): """ Default weight is 1. None label is replaced by eps. Default olabel of None is replaced by label. """ self.filename = None if label is None: label = self.eps if olabel is None: olabel = label if weight is None: weight = 1 weight = -math.log(weight) if weight != 0 else self.zero label_id = self.word_to_ilabel_map[label] olabel_id = self.word_to_olabel_map[olabel] result = self._lib.fst__add_arc(self.native_obj, int(src_state), int(dst_state), int(label_id), int(olabel_id), float(weight)) if not result: raise KaldiError("Failed fst__add_arc") self.num_arcs += 1 def compute_hash(self, dependencies_seed_hash_str='0'*32): hash_p = _ffi.new('char[]', 33) # Length of MD5 hex string + null terminator result = self._lib.fst__compute_md5(self.native_obj, hash_p, encode(dependencies_seed_hash_str)) if not result: raise KaldiError("Failed fst__compute_md5") hash_str = decode(_ffi.string(hash_p)) self.filename = hash_str + '.fst' return hash_str #################################################################################################################### def has_path(self): """ Returns True iff there is a path (from start state to a final state). Uses BFS. Assumes can nonterminals succeed. """ result = self._lib.fst__has_path(self.native_obj) return result def has_eps_path(self, path_src_state, path_dst_state, eps_like_labels=frozenset()): """ Returns True iff there is a epsilon-like-only path from src_state to dst_state. Uses BFS. Does not follow nonterminals! """ assert not eps_like_labels result = self._lib.fst__has_eps_path(self.native_obj, path_src_state, path_dst_state) return result def does_match(self, target_words, wildcard_nonterms=(), include_silent=False, output_max_length=1024): """ Returns the olabels on a matching path if there is one, False if not. Uses BFS. Wildcard accepts zero or more words. """ # FIXME: do in decoder! assert frozenset(wildcard_nonterms) == self.wildcard_nonterms output_p = _ffi.new('int32_t[]', output_max_length) output_len_p = _ffi.new('int32_t*', output_max_length) target_labels = [self.word_to_ilabel_map[word] for word in target_words] result = self._lib.fst__does_match(self.native_obj, len(target_labels), target_labels, output_p, output_len_p) if output_len_p[0] > output_max_length: raise KaldiError("fst__does_match needed too much output length") if result: return tuple(self.olabel_to_word_map[symbol] for symbol in output_p[0:output_len_p[0]] if include_silent or symbol not in self.silent_olabels) return False #################################################################################################################### def write_file(self, fst_filename): result = self._lib.fst__write_file(self.native_obj, encode(fst_filename)) if not result: raise KaldiError("Failed fst__write_file") def write_file_const(self, fst_filename): result = self._lib.fst__write_file_const(self.native_obj, encode(fst_filename)) if not result: raise KaldiError("Failed fst__write_file") def print(self, fst_filename=None): result = self._lib.fst__print(self.native_obj, (encode(fst_filename) if fst_filename is not None else _ffi.NULL)) if not result: raise KaldiError("Failed fst__print") @classmethod def load_file(cls, fst_filename): cls.init_ffi() native_obj = cls._lib.fst__load_file(encode(fst_filename)) if not native_obj: raise KaldiError("Failed fst__load_file") # FIXME: memory leak possible? return native_obj @classmethod def compile_text(cls, fst_text, isymbols_filename, osymbols_filename): cls.init_ffi() native_obj = cls._lib.fst__compile_text(encode(fst_text), encode(isymbols_filename), encode(osymbols_filename)) if not native_obj: raise KaldiError("Failed fst__compile_text") # FIXME: memory leak possible? return native_obj ######################################################################################################################## class SymbolTable(object): def __init__(self, filename=None): self.word_to_id_map = dict() self.id_to_word_map = dict() self.max_term_word_id = -1 if filename is not None: self.load_text_file(filename) def load_text_file(self, filename): with open(filename, 'r', encoding='utf-8') as file: word_id_pairs = [line.strip().split() for line in file] self.word_to_id_map.clear() self.id_to_word_map.clear() self.word_to_id_map.update({ word: int(id) for (word, id) in word_id_pairs }) self.id_to_word_map.update({ id: word for (word, id) in self.word_to_id_map.items() }) self.max_term_word_id = max(id for (word, id) in self.word_to_id_map.items() if not word.startswith('#nonterm')) def add_word(self, word, id=None): if id is None: self.max_term_word_id += 1 id = self.max_term_word_id else: id = int(id) self.word_to_id_map[word] = id self.id_to_word_map[id] = word words = property(lambda self: self.word_to_id_map.keys()) def __contains__(self, word): return (word in self.word_to_id_map) ================================================ FILE: kaldi_active_grammar/wrapper.py ================================================ # # This file is part of kaldi-active-grammar. # (c) Copyright 2019 by David Zurow # Licensed under the AGPL-3.0; see LICENSE.txt file. # """ Wrapper classes for Kaldi """ import argparse, json, os.path, sys from io import open, StringIO from six.moves import zip import numpy as np from . import _log, KaldiError from .ffi import FFIObject, _ffi, decode as de, encode as en from .utils import clock, find_file, show_donation_message, symbol_table_lookup from .wfst import NativeWFST import kaldi_active_grammar.defaults as defaults _log = _log.getChild('wrapper') _log_library = _log.getChild('library') ######################################################################################################################## class KaldiDecoderBase(FFIObject): """docstring for KaldiDecoderBase""" def __init__(self): super(KaldiDecoderBase, self).__init__() show_donation_message() self.sample_rate = 16000 self.num_channels = 1 self.bytes_per_kaldi_frame = self.kaldi_frame_num_to_audio_bytes(1) self._reset_decode_time() def _reset_decode_time(self): self._decode_time = 0 self._decode_real_time = 0 self._decode_times = [] def _start_decode_time(self, num_frames): self.decode_start_time = clock() self._decode_real_time += 1000.0 * num_frames / self.sample_rate def _stop_decode_time(self, finalize=False): this = (clock() - self.decode_start_time) * 1000.0 self._decode_time += this self._decode_times.append(this) if finalize: rtf = 1.0 * self._decode_time / self._decode_real_time if self._decode_real_time != 0 else float('nan') pct = 100.0 * this / self._decode_time if self._decode_time != 0 else 100 _log.log(15, "decoded at %.2f RTF, for %d ms audio, spending %d ms, of which %d ms (%.0f%%) in finalization", rtf, self._decode_real_time, self._decode_time, this, pct) _log.log(13, " decode times: %s", ' '.join("%d" % t for t in self._decode_times)) self._reset_decode_time() def kaldi_frame_num_to_audio_bytes(self, kaldi_frame_num): kaldi_frame_length_ms = 30 sample_size_bytes = 2 * self.num_channels return int(kaldi_frame_num * kaldi_frame_length_ms * self.sample_rate / 1000 * sample_size_bytes) def audio_bytes_to_s(self, audio_bytes): sample_size_bytes = 2 * self.num_channels return 1.0 * audio_bytes // sample_size_bytes / self.sample_rate ######################################################################################################################## class KaldiGmmDecoder(KaldiDecoderBase): """docstring for KaldiGmmDecoder""" _library_header_text = """ void* gmm__init(float beam, int32_t max_active, int32_t min_active, float lattice_beam, char* word_syms_filename_cp, char* fst_in_str_cp, char* config_cp); bool gmm__decode(void* model_vp, float samp_freq, int32_t num_frames, float* frames, bool finalize); bool gmm__get_output(void* model_vp, char* output, int32_t output_length, double* likelihood_p); """ def __init__(self, graph_dir=None, words_file=None, graph_file=None, model_conf_file=None): super(KaldiGmmDecoder, self).__init__() if words_file is None and graph_dir is not None: words_file = graph_dir + r"graph\words.txt" if graph_file is None and graph_dir is not None: graph_file = graph_dir + r"graph\HCLG.fst" self.words_file = os.path.normpath(words_file) self.graph_file = os.path.normpath(graph_file) self.model_conf_file = os.path.normpath(model_conf_file) self._model = self._lib.gmm__init(7.0, 7000, 200, 8.0, words_file, graph_file, model_conf_file) if not self._model: raise KaldiError("failed gmm__init") self.sample_rate = 16000 def decode(self, frames, finalize, grammars_activity=None): if not isinstance(frames, np.ndarray): frames = np.frombuffer(frames, np.int16) frames = frames.astype(np.float32) frames_char = _ffi.from_buffer(frames) frames_float = _ffi.cast('float *', frames_char) self._start_decode_time(len(frames)) result = self._lib.gmm__decode(self._model, self.sample_rate, len(frames), frames_float, finalize) self._stop_decode_time(finalize) if not result: raise RuntimeError("decoding error") return finalize def get_output(self, output_max_length=4*1024): output_p = _ffi.new('char[]', output_max_length) likelihood_p = _ffi.new('double *') result = self._lib.gmm__get_output(self._model, output_p, output_max_length, likelihood_p) output_str = _ffi.string(output_p) info = { 'likelihood': likelihood_p[0], } return output_str, info ######################################################################################################################## class KaldiOtfGmmDecoder(KaldiDecoderBase): """docstring for KaldiOtfGmmDecoder""" _library_header_text = """ void* gmm_otf__init(float beam, int32_t max_active, int32_t min_active, float lattice_beam, char* word_syms_filename_cp, char* config_cp, char* hcl_fst_filename_cp, char** grammar_fst_filenames_cp, int32_t grammar_fst_filenames_len); bool gmm_otf__add_grammar_fst(void* model_vp, char* grammar_fst_filename_cp); bool gmm_otf__decode(void* model_vp, float samp_freq, int32_t num_frames, float* frames, bool finalize, bool* grammars_activity, int32_t grammars_activity_size); bool gmm_otf__get_output(void* model_vp, char* output, int32_t output_length, double* likelihood_p); """ def __init__(self, graph_dir=None, words_file=None, model_conf_file=None, hcl_fst_file=None, grammar_fst_files=None): super(KaldiOtfGmmDecoder, self).__init__() if words_file is None and graph_dir is not None: words_file = graph_dir + r"graph\words.txt" if hcl_fst_file is None and graph_dir is not None: hcl_fst_file = graph_dir + r"graph\HCLr.fst" if grammar_fst_files is None and graph_dir is not None: grammar_fst_files = [graph_dir + r"graph\Gr.fst"] self.words_file = os.path.normpath(words_file) self.model_conf_file = os.path.normpath(model_conf_file) self.hcl_fst_file = os.path.normpath(hcl_fst_file) grammar_fst_filenames_cps = [_ffi.new('char[]', os.path.normpath(f)) for f in grammar_fst_files] grammar_fst_filenames_cp = _ffi.new('char*[]', grammar_fst_filenames_cps) self._model = self._lib.gmm_otf__init(7.0, 7000, 200, 8.0, words_file, model_conf_file, hcl_fst_file, _ffi.cast('char**', grammar_fst_filenames_cp), len(grammar_fst_files)) if not self._model: raise KaldiError("failed gmm_otf__init") self.sample_rate = 16000 self.num_grammars = len(grammar_fst_files) def add_grammar_fst(self, grammar_fst_file): grammar_fst_file = os.path.normpath(grammar_fst_file) _log.log(8, "%s: adding grammar_fst_file: %s", self, grammar_fst_file) result = self._lib.gmm_otf__add_grammar_fst(self._model, grammar_fst_file) if not result: raise KaldiError("error adding grammar") self.num_grammars += 1 def decode(self, frames, finalize, grammars_activity=None): # grammars_activity = [True] * self.num_grammars # grammars_activity = np.random.choice([True, False], len(grammars_activity)).tolist(); print grammars_activity; time.sleep(5) if grammars_activity is None: grammars_activity = [] else: _log.debug("decode: grammars_activity = %s", ''.join('1' if a else '0' for a in grammars_activity)) # if len(grammars_activity) != self.num_grammars: # raise KaldiError("wrong len(grammars_activity)") if not isinstance(frames, np.ndarray): frames = np.frombuffer(frames, np.int16) frames = frames.astype(np.float32) frames_char = _ffi.from_buffer(frames) frames_float = _ffi.cast('float *', frames_char) self._start_decode_time(len(frames)) result = self._lib.gmm_otf__decode(self._model, self.sample_rate, len(frames), frames_float, finalize, grammars_activity, len(grammars_activity)) self._stop_decode_time(finalize) if not result: raise KaldiError("decoding error") return finalize def get_output(self, output_max_length=4*1024): output_p = _ffi.new('char[]', output_max_length) likelihood_p = _ffi.new('double *') result = self._lib.gmm_otf__get_output(self._model, output_p, output_max_length, likelihood_p) output_str = _ffi.string(output_p) info = { 'likelihood': likelihood_p[0], } return output_str, info ######################################################################################################################## class KaldiNNet3Decoder(KaldiDecoderBase): """ Abstract base class for nnet3 decoders. """ _library_header_text = """ DRAGONFLY_API bool nnet3_base__load_lexicon(void* model_vp, char* word_syms_filename_cp, char* word_align_lexicon_filename_cp); DRAGONFLY_API bool nnet3_base__save_adaptation_state(void* model_vp); DRAGONFLY_API bool nnet3_base__reset_adaptation_state(void* model_vp); DRAGONFLY_API bool nnet3_base__get_word_align(void* model_vp, int32_t* times_cp, int32_t* lengths_cp, int32_t num_words); DRAGONFLY_API bool nnet3_base__decode(void* model_vp, float samp_freq, int32_t num_samples, float* samples, bool finalize, bool save_adaptation_state); DRAGONFLY_API bool nnet3_base__get_output(void* model_vp, char* output, int32_t output_max_length, float* likelihood_p, float* am_score_p, float* lm_score_p, float* confidence_p, float* expected_error_rate_p); DRAGONFLY_API bool nnet3_base__set_lm_prime_text(void* model_vp, char* prime_cp); """ def __init__(self, model_dir, tmp_dir, words_file=None, word_align_lexicon_file=None, max_num_rules=None, save_adaptation_state=False): super(KaldiNNet3Decoder, self).__init__() model_dir = os.path.normpath(model_dir) if words_file is None: words_file = find_file(model_dir, 'words.txt') if word_align_lexicon_file is None: word_align_lexicon_file = find_file(model_dir, 'align_lexicon.int', required=False) mfcc_conf_file = find_file(model_dir, 'mfcc_hires.conf') if mfcc_conf_file is None: mfcc_conf_file = find_file(model_dir, 'mfcc.conf') # FIXME: warning? model_file = find_file(model_dir, 'final.mdl') self.model_dir = model_dir self.words_file = os.path.normpath(words_file) self.word_align_lexicon_file = os.path.normpath(word_align_lexicon_file) if word_align_lexicon_file is not None else None self.mfcc_conf_file = os.path.normpath(mfcc_conf_file) self.model_file = os.path.normpath(model_file) self.ie_config = self._read_ie_conf_file(model_dir, find_file(model_dir, 'ivector_extractor.conf')) self.verbosity = (10 - _log_library.getEffectiveLevel()) if _log_library.isEnabledFor(10) else -1 self.max_num_rules = int(max_num_rules) if max_num_rules is not None else None self._saving_adaptation_state = save_adaptation_state self.config_dict = { 'model_dir': self.model_dir, 'mfcc_config_filename': self.mfcc_conf_file, 'ivector_extraction_config_json': self.ie_config, 'model_filename': self.model_file, 'word_syms_filename': self.words_file, 'word_align_lexicon_filename': self.word_align_lexicon_file or '', } if self.max_num_rules is not None: self.config_dict.update(max_num_rules=self.max_num_rules) def _read_ie_conf_file(self, model_dir, old_filename, search=True): """ Read ivector_extractor.conf file, converting relative paths to absolute paths for current configuration, returning dict of config. """ options_with_path = { '--splice-config': 'conf/splice.conf', '--cmvn-config': 'conf/online_cmvn.conf', '--lda-matrix': 'ivector_extractor/final.mat', '--global-cmvn-stats': 'ivector_extractor/global_cmvn.stats', '--diag-ubm': 'ivector_extractor/final.dubm', '--ivector-extractor': 'ivector_extractor/final.ie', } def convert_path(key, value): if not search: return os.path.join(model_dir, options_with_path[key]) else: return find_file(model_dir, os.path.basename(options_with_path[key]), required=True) options_converters = { '--splice-config': convert_path, '--cmvn-config': convert_path, '--lda-matrix': convert_path, '--global-cmvn-stats': convert_path, '--diag-ubm': convert_path, '--ivector-extractor': convert_path, '--ivector-period': lambda key, value: (float(value) if '.' in value else int(value)), '--max-count': lambda key, value: (float(value) if '.' in value else int(value)), '--max-remembered-frames': lambda key, value: (float(value) if '.' in value else int(value)), '--min-post': lambda key, value: (float(value) if '.' in value else int(value)), '--num-gselect': lambda key, value: (float(value) if '.' in value else int(value)), '--posterior-scale': lambda key, value: (float(value) if '.' in value else int(value)), '--online-cmvn-iextractor': lambda key, value: (True if value in ['true'] else False), } config = dict() with open(old_filename, 'r', encoding='utf-8') as old_file: for line in old_file: key, value = line.strip().split('=', 1) value = options_converters[key](key, value) assert key.startswith('--') key = key[2:] config[key] = value return config saving_adaptation_state = property(lambda self: self._saving_adaptation_state, doc="Whether currently to save updated adaptation state at end of utterance") @saving_adaptation_state.setter def saving_adaptation_state(self, value): self._saving_adaptation_state = value def load_lexicon(self, words_file=None, word_align_lexicon_file=None): """ Only necessary when you update the lexicon after initialization. """ if words_file is None: words_file = self.words_file if word_align_lexicon_file is None: word_align_lexicon_file = self.word_align_lexicon_file result = self._lib.nnet3_base__load_lexicon(self._model, en(words_file), en(word_align_lexicon_file)) if not result: raise KaldiError("error loading lexicon (%r, %r)" % (words_file, word_align_lexicon_file)) def save_adaptation_state(self): result = self._lib.nnet3_base__save_adaptation_state(self._model) if not result: raise KaldiError("save_adaptation_state error") def reset_adaptation_state(self): result = self._lib.nnet3_base__reset_adaptation_state(self._model) if not result: raise KaldiError("reset_adaptation_state error") def get_output(self, output_max_length=4*1024): output_p = _ffi.new('char[]', output_max_length) likelihood_p = _ffi.new('float *') am_score_p = _ffi.new('float *') lm_score_p = _ffi.new('float *') confidence_p = _ffi.new('float *') expected_error_rate_p = _ffi.new('float *') result = self._lib.nnet3_base__get_output(self._model, output_p, output_max_length, likelihood_p, am_score_p, lm_score_p, confidence_p, expected_error_rate_p) if not result: raise KaldiError("get_output error") output_str = de(_ffi.string(output_p)) info = { 'likelihood': likelihood_p[0], 'am_score': am_score_p[0], 'lm_score': lm_score_p[0], 'confidence': confidence_p[0], 'expected_error_rate': expected_error_rate_p[0], } _log.log(7, "get_output: %r %s", output_str, info) return output_str, info def get_word_align(self, output): """Returns tuple of tuples: words (including nonterminals but not eps), each's time (in bytes), and each's length (in bytes).""" words = output.split() num_words = len(words) kaldi_frame_times_p = _ffi.new('int32_t[]', num_words) kaldi_frame_lengths_p = _ffi.new('int32_t[]', num_words) result = self._lib.nnet3_base__get_word_align(self._model, kaldi_frame_times_p, kaldi_frame_lengths_p, num_words) if not result: raise KaldiError("get_word_align error") times = [kaldi_frame_num * self.bytes_per_kaldi_frame for kaldi_frame_num in kaldi_frame_times_p] lengths = [kaldi_frame_num * self.bytes_per_kaldi_frame for kaldi_frame_num in kaldi_frame_lengths_p] return tuple(zip(words, times, lengths)) def set_lm_prime_text(self, prime_text): prime_text = prime_text.strip() result = self._lib.nnet3_base__set_lm_prime_text(self._model, en(prime_text)) if not result: raise KaldiError("error setting prime text %r" % prime_text) ######################################################################################################################## class KaldiPlainNNet3Decoder(KaldiNNet3Decoder): """docstring for KaldiPlainNNet3Decoder""" _library_header_text = KaldiNNet3Decoder._library_header_text + """ DRAGONFLY_API void* nnet3_plain__construct(char* model_dir_cp, char* config_str_cp, int32_t verbosity); DRAGONFLY_API bool nnet3_plain__destruct(void* model_vp); DRAGONFLY_API bool nnet3_plain__decode(void* model_vp, float samp_freq, int32_t num_samples, float* samples, bool finalize, bool save_adaptation_state); """ def __init__(self, fst_file=None, config=None, **kwargs): super(KaldiPlainNNet3Decoder, self).__init__(**kwargs) if fst_file is None: fst_file = find_file(self.model_dir, defaults.DEFAULT_PLAIN_DICTATION_HCLG_FST_FILENAME, required=True) fst_file = os.path.normpath(fst_file) self.config_dict.update({ 'decode_fst_filename': fst_file, }) if config: self.config_dict.update(config) _log.debug("config_dict: %s", self.config_dict) self._model = self._lib.nnet3_plain__construct(en(self.model_dir), en(json.dumps(self.config_dict)), self.verbosity) if not self._model: raise KaldiError("failed nnet3_plain__construct") def destroy(self): if self._model: result = self._lib.nnet3_plain__destruct(self._model) if not result: raise KaldiError("failed nnet3_plain__destruct") self._model = None def decode(self, frames, finalize): """Continue decoding with given new audio data.""" if not isinstance(frames, np.ndarray): frames = np.frombuffer(frames, np.int16) frames = frames.astype(np.float32) frames_char = _ffi.from_buffer(frames) frames_float = _ffi.cast('float *', frames_char) self._start_decode_time(len(frames)) result = self._lib.nnet3_plain__decode(self._model, self.sample_rate, len(frames), frames_float, finalize, self._saving_adaptation_state) self._stop_decode_time(finalize) if not result: raise KaldiError("decoding error") return finalize ######################################################################################################################## class KaldiAgfNNet3Decoder(KaldiNNet3Decoder): """docstring for KaldiAgfNNet3Decoder""" _library_header_text = KaldiNNet3Decoder._library_header_text + """ DRAGONFLY_API void* nnet3_agf__construct(char* model_dir_cp, char* config_str_cp, int32_t verbosity); DRAGONFLY_API bool nnet3_agf__destruct(void* model_vp); DRAGONFLY_API int32_t nnet3_agf__add_grammar_fst(void* model_vp, void* grammar_fst_cp); DRAGONFLY_API int32_t nnet3_agf__add_grammar_fst_file(void* model_vp, char* grammar_fst_filename_cp); DRAGONFLY_API bool nnet3_agf__reload_grammar_fst(void* model_vp, int32_t grammar_fst_index, void* grammar_fst_cp); DRAGONFLY_API bool nnet3_agf__reload_grammar_fst_file(void* model_vp, int32_t grammar_fst_index, char* grammar_fst_filename_cp); DRAGONFLY_API bool nnet3_agf__remove_grammar_fst(void* model_vp, int32_t grammar_fst_index); DRAGONFLY_API bool nnet3_agf__decode(void* model_vp, float samp_freq, int32_t num_frames, float* frames, bool finalize, bool* grammars_activity_cp, int32_t grammars_activity_cp_size, bool save_adaptation_state); """ def __init__(self, *, top_fst=None, dictation_fst_file=None, config=None, **kwargs): super(KaldiAgfNNet3Decoder, self).__init__(**kwargs) phones_file = find_file(self.model_dir, 'phones.txt') nonterm_phones_offset = symbol_table_lookup(phones_file, '#nonterm_bos') if nonterm_phones_offset is None: raise KaldiError("cannot find #nonterm_bos symbol in phones.txt") rules_phones_offset = symbol_table_lookup(phones_file, '#nonterm:rule0') if rules_phones_offset is None: raise KaldiError("cannot find #nonterm:rule0 symbol in phones.txt") dictation_phones_offset = symbol_table_lookup(phones_file, '#nonterm:dictation') if dictation_phones_offset is None: raise KaldiError("cannot find #nonterm:dictation symbol in phones.txt") self.config_dict.update({ 'nonterm_phones_offset': nonterm_phones_offset, 'rules_phones_offset': rules_phones_offset, 'dictation_phones_offset': dictation_phones_offset, 'dictation_fst_filename': os.path.normpath(dictation_fst_file) if dictation_fst_file is not None else '', }) if isinstance(top_fst, NativeWFST): self.config_dict.update({'top_fst': int(_ffi.cast("uint64_t", top_fst.compiled_native_obj))}) elif isinstance(top_fst, str): self.config_dict.update({'top_fst_filename': os.path.normpath(top_fst)}) else: raise KaldiError("unrecognized top_fst type") if config: self.config_dict.update(config) _log.debug("config_dict: %s", self.config_dict) self._model = self._lib.nnet3_agf__construct(en(self.model_dir), en(json.dumps(self.config_dict)), self.verbosity) if not self._model: raise KaldiError("failed nnet3_agf__construct") self.num_grammars = 0 def destroy(self): if self._model: result = self._lib.nnet3_agf__destruct(self._model) if not result: raise KaldiError("failed nnet3_agf__destruct") self._model = None def add_grammar_fst(self, grammar_fst): _log.log(8, "%s: adding grammar_fst: %r", self, grammar_fst) if isinstance(grammar_fst, NativeWFST): grammar_fst_index = self._lib.nnet3_agf__add_grammar_fst(self._model, grammar_fst.compiled_native_obj) elif isinstance(grammar_fst, str): grammar_fst_index = self._lib.nnet3_agf__add_grammar_fst_file(self._model, en(os.path.normpath(grammar_fst))) else: raise KaldiError("unrecognized grammar_fst type") if grammar_fst_index < 0: raise KaldiError("error adding grammar %r" % grammar_fst) assert grammar_fst_index == self.num_grammars, "add_grammar_fst allocated invalid grammar_fst_index" self.num_grammars += 1 return grammar_fst_index def reload_grammar_fst(self, grammar_fst_index, grammar_fst): _log.debug("%s: reloading grammar_fst_index: #%s %r", self, grammar_fst_index, grammar_fst) if isinstance(grammar_fst, NativeWFST): result = self._lib.nnet3_agf__reload_grammar_fst(self._model, grammar_fst_index, grammar_fst.compiled_native_obj) elif isinstance(grammar_fst, str): result = self._lib.nnet3_agf__reload_grammar_fst_file(self._model, grammar_fst_index, en(os.path.normpath(grammar_fst))) else: raise KaldiError("unrecognized grammar_fst type") if not result: raise KaldiError("error reloading grammar #%s %r" % (grammar_fst_index, grammar_fst)) def remove_grammar_fst(self, grammar_fst_index): _log.debug("%s: removing grammar_fst_index: %s", self, grammar_fst_index) result = self._lib.nnet3_agf__remove_grammar_fst(self._model, grammar_fst_index) if not result: raise KaldiError("error removing grammar #%s" % grammar_fst_index) self.num_grammars -= 1 def decode(self, frames, finalize, grammars_activity=None): """Continue decoding with given new audio data.""" # grammars_activity = [True] * self.num_grammars # grammars_activity = np.random.choice([True, False], len(grammars_activity)).tolist(); print grammars_activity; time.sleep(5) if grammars_activity is None: grammars_activity = [] else: # Start of utterance _log.log(5, "decode: grammars_activity = %s", ''.join('1' if a else '0' for a in grammars_activity)) if len(grammars_activity) != self.num_grammars: _log.error("wrong len(grammars_activity) = %d != %d = num_grammars" % (len(grammars_activity), self.num_grammars)) if not isinstance(frames, np.ndarray): frames = np.frombuffer(frames, np.int16) frames = frames.astype(np.float32) frames_char = _ffi.from_buffer(frames) frames_float = _ffi.cast('float *', frames_char) self._start_decode_time(len(frames)) result = self._lib.nnet3_agf__decode(self._model, self.sample_rate, len(frames), frames_float, finalize, grammars_activity, len(grammars_activity), self._saving_adaptation_state) self._stop_decode_time(finalize) if not result: raise KaldiError("decoding error") return finalize ######################################################################################################################## class KaldiAgfCompiler(FFIObject): _library_header_text = """ DRAGONFLY_API void* nnet3_agf__construct_compiler(char* config_str_cp); DRAGONFLY_API bool nnet3_agf__destruct_compiler(void* compiler_vp); DRAGONFLY_API void* nnet3_agf__compile_graph(void* compiler_vp, char* config_str_cp, void* grammar_fst_cp, bool return_graph); DRAGONFLY_API void* nnet3_agf__compile_graph_text(void* compiler_vp, char* config_str_cp, char* grammar_fst_text_cp, bool return_graph); DRAGONFLY_API void* nnet3_agf__compile_graph_file(void* compiler_vp, char* config_str_cp, char* grammar_fst_filename_cp, bool return_graph); """ def __init__(self, config): super(KaldiAgfCompiler, self).__init__() self._compiler = self._lib.nnet3_agf__construct_compiler(en(json.dumps(config))) if not self._compiler: raise KaldiError("failed nnet3_agf__construct_compiler") def destroy(self): if self._compiler: result = self._lib.nnet3_agf__destruct_compiler(self._compiler) if not result: raise KaldiError("failed nnet3_agf__destruct_compiler") self._compiler = None def compile_graph(self, config, grammar_fst=None, grammar_fst_text=None, grammar_fst_file=None, return_graph=False): if 1 != sum(int(g is not None) for g in [grammar_fst, grammar_fst_text, grammar_fst_file]): raise ValueError("must pass exactly one grammar") if grammar_fst is not None: _log.log(5, "compile_graph:\n config=%r\n grammar_fst=%r", config, grammar_fst) result = self._lib.nnet3_agf__compile_graph(self._compiler, en(json.dumps(config)), grammar_fst.native_obj, return_graph) return result if grammar_fst_text is not None: _log.log(5, "compile_graph:\n config=%r\n grammar_fst_text:\n%s", config, grammar_fst_text) result = self._lib.nnet3_agf__compile_graph_text(self._compiler, en(json.dumps(config)), en(grammar_fst_text), return_graph) return result if grammar_fst_file is not None: _log.log(5, "compile_graph:\n config=%r\n grammar_fst_file=%r", config, grammar_fst_file) result = self._lib.nnet3_agf__compile_graph_file(self._compiler, en(json.dumps(config)), en(grammar_fst_file), return_graph) return result ######################################################################################################################## class KaldiLafNNet3Decoder(KaldiNNet3Decoder): """docstring for KaldiLafNNet3Decoder""" _library_header_text = KaldiNNet3Decoder._library_header_text + """ DRAGONFLY_API void* nnet3_laf__construct(char* model_dir_cp, char* config_str_cp, int32_t verbosity); DRAGONFLY_API bool nnet3_laf__destruct(void* model_vp); DRAGONFLY_API int32_t nnet3_laf__add_grammar_fst(void* model_vp, void* grammar_fst_cp); DRAGONFLY_API int32_t nnet3_laf__add_grammar_fst_text(void* model_vp, char* grammar_fst_cp); DRAGONFLY_API bool nnet3_laf__reload_grammar_fst(void* model_vp, int32_t grammar_fst_index, void* grammar_fst_cp); DRAGONFLY_API bool nnet3_laf__remove_grammar_fst(void* model_vp, int32_t grammar_fst_index); DRAGONFLY_API bool nnet3_laf__decode(void* model_vp, float samp_freq, int32_t num_frames, float* frames, bool finalize, bool* grammars_activity_cp, int32_t grammars_activity_cp_size, bool save_adaptation_state); """ def __init__(self, dictation_fst_file=None, config=None, **kwargs): super(KaldiLafNNet3Decoder, self).__init__(**kwargs) self.config_dict.update({ 'hcl_fst_filename': find_file(self.model_dir, 'HCLr.fst'), 'disambig_tids_filename': find_file(self.model_dir, 'disambig_tid.int'), 'relabel_ilabels_filename': find_file(self.model_dir, 'relabel_ilabels.int'), 'word_syms_relabeled_filename': find_file(self.model_dir, 'words.relabeled.txt', required=True), 'dictation_fst_filename': dictation_fst_file or '', }) if config: self.config_dict.update(config) _log.debug("config_dict: %s", self.config_dict) self._model = self._lib.nnet3_laf__construct(en(self.model_dir), en(json.dumps(self.config_dict)), self.verbosity) if not self._model: raise KaldiError("failed nnet3_laf__construct") self.num_grammars = 0 def destroy(self): if self._model: result = self._lib.nnet3_laf__destruct(self._model) if not result: raise KaldiError("failed nnet3_laf__destruct") self._model = None def add_grammar_fst(self, grammar_fst): _log.log(8, "%s: adding grammar_fst: %r", self, grammar_fst) grammar_fst_index = self._lib.nnet3_laf__add_grammar_fst(self._model, grammar_fst.native_obj) if grammar_fst_index < 0: raise KaldiError("error adding grammar %r" % grammar_fst) assert grammar_fst_index == self.num_grammars, "add_grammar_fst allocated invalid grammar_fst_index" self.num_grammars += 1 return grammar_fst_index def add_grammar_fst_text(self, grammar_fst_text): assert grammar_fst_text _log.log(8, "%s: adding grammar_fst_text: %r", self, grammar_fst_text[:512]) grammar_fst_index = self._lib.nnet3_laf__add_grammar_fst_text(self._model, en(grammar_fst_text)) if grammar_fst_index < 0: raise KaldiError("error adding grammar %r" % grammar_fst_text[:512]) assert grammar_fst_index == self.num_grammars, "add_grammar_fst allocated invalid grammar_fst_index" self.num_grammars += 1 return grammar_fst_index def reload_grammar_fst(self, grammar_fst_index, grammar_fst): _log.debug("%s: reloading grammar_fst_index: #%s %r", self, grammar_fst_index, grammar_fst) result = self._lib.nnet3_laf__reload_grammar_fst(self._model, grammar_fst_index, grammar_fst.native_obj) if not result: raise KaldiError("error reloading grammar #%s %r" % (grammar_fst_index, grammar_fst)) def remove_grammar_fst(self, grammar_fst_index): _log.debug("%s: removing grammar_fst_index: %s", self, grammar_fst_index) result = self._lib.nnet3_laf__remove_grammar_fst(self._model, grammar_fst_index) if not result: raise KaldiError("error removing grammar #%s" % grammar_fst_index) self.num_grammars -= 1 def decode(self, frames, finalize, grammars_activity=None): """Continue decoding with given new audio data.""" # grammars_activity = [True] * self.num_grammars # grammars_activity = np.random.choice([True, False], len(grammars_activity)).tolist(); print grammars_activity; time.sleep(5) if grammars_activity is None: grammars_activity = [] else: # Start of utterance _log.log(5, "decode: grammars_activity = %s", ''.join('1' if a else '0' for a in grammars_activity)) if len(grammars_activity) != self.num_grammars: _log.error("wrong len(grammars_activity) = %d != %d = num_grammars" % (len(grammars_activity), self.num_grammars)) if not isinstance(frames, np.ndarray): frames = np.frombuffer(frames, np.int16) frames = frames.astype(np.float32) frames_char = _ffi.from_buffer(frames) frames_float = _ffi.cast('float *', frames_char) self._start_decode_time(len(frames)) result = self._lib.nnet3_laf__decode(self._model, self.sample_rate, len(frames), frames_float, finalize, grammars_activity, len(grammars_activity), self._saving_adaptation_state) self._stop_decode_time(finalize) if not result: raise KaldiError("decoding error") return finalize ######################################################################################################################## class KaldiModelBuildUtils(FFIObject): _library_header_text = """ DRAGONFLY_API bool utils__build_L_disambig(char* lexicon_fst_text_cp, char* isymbols_file_cp, char* osymbols_file_cp, char* wdisambig_phones_file_cp, char* wdisambig_words_file_cp, char* fst_out_file_cp); """ @classmethod def build_L_disambig(cls, lexicon_fst_text_bytes, phones_file, words_file, wdisambig_phones_file, wdisambig_words_file, fst_out_file): cls.init_ffi() result = cls._lib.utils__build_L_disambig(lexicon_fst_text_bytes, en(phones_file), en(words_file), en(wdisambig_phones_file), en(wdisambig_words_file), en(fst_out_file)) if not result: raise KaldiError("failed utils__build_L_disambig") @staticmethod def make_lexicon_fst(**kwargs): try: from .kaldi.make_lexicon_fst import main old_stdout = sys.stdout sys.stdout = output = StringIO() main(argparse.Namespace(**kwargs)) return output.getvalue() finally: sys.stdout = old_stdout ================================================ FILE: pyproject.toml ================================================ [build-system] requires = ["setuptools", "wheel", "scikit-build", "cmake", "ninja"] [tool.pytest.ini_options] minversion = "8.0" testpaths = ["tests"] addopts = [ "--import-mode=importlib", "-ra", # extra summary for skips/xfails "--strict-markers", # fail on unknown markers "--strict-config", # fail on bad config # "--cov=kaldi_active_grammar", "--no-cov-on-fail", "--cov-branch", "--cov-report=term-missing", ] xfail_strict = true ================================================ FILE: requirements-build.txt ================================================ cmake ninja scikit-build>=0.10.0 setuptools wheel ================================================ FILE: requirements-editable.txt ================================================ -e . ================================================ FILE: requirements-test.txt ================================================ piper-tts~=1.0 pytest>=8.0 pytest-cov ================================================ FILE: setup.cfg ================================================ [metadata] # This includes the license file(s) in the wheel. # https://wheel.readthedocs.io/en/stable/user_guide.html#including-license-files-in-the-generated-wheel-file license_files = LICENSE.txt [bdist_wheel] # This flag says to generate wheels that support both Python 2 and Python # 3. If your code will not run unchanged on both Python 2 and 3, you will # need to generate separate wheels for each Python version that you # support. Removing this line (or setting universal to 0) will prevent # bdist_wheel from trying to make a universal wheel. For more see: # https://packaging.python.org/guides/distributing-packages-using-setuptools/#wheels # universal=1 ================================================ FILE: setup.py ================================================ """A setuptools based setup module. See: https://packaging.python.org/guides/distributing-packages-using-setuptools/ https://github.com/pypa/sampleproject """ # Always prefer setuptools over distutils from setuptools import find_packages import datetime import os import platform import re # Optionally skip native code build (expecting libraries to be manually/externally placed correctly prior) by using standard setuptools; otherwise build native code with scikit-build if os.environ.get('KALDIAG_BUILD_SKIP_NATIVE') or os.environ.get('KALDIAG_SETUP_RAW'): from setuptools import setup else: from skbuild import setup import site, sys site.ENABLE_USER_SITE = bool("--user" in sys.argv[1:]) # Fix pip bug breaking editable install to user directory: https://github.com/pypa/pip/issues/7953 # Force wheel to be platform-specific (needed due to manually-loaded native libraries) # https://stackoverflow.com/questions/45150304/how-to-force-a-python-wheel-to-be-platform-specific-when-building-it # https://github.com/Yelp/dumb-init/blob/48db0c0d0ecb4598d1a6400710445b85d67616bf/setup.py#L11-L27 # https://github.com/google/or-tools/issues/616#issuecomment-371480314 if True: from wheel.bdist_wheel import bdist_wheel as bdist_wheel class bdist_wheel_impure(bdist_wheel): def finalize_options(self): super().finalize_options() # Mark us as not a pure python package: we contain platform-specific native libraries, even though no CPython extensions self.root_is_pure = False def get_tag(self): python, abi, plat = super().get_tag() # Mark us as python-version-agnostic (py3), and python-ABI-agnostic (none), since we contain no CPython extensions python, abi = 'py3', 'none' # For MacOS, prevent mistakenly marking as universal2 wheel (since we compile our native libraries as either x86_64 or arm64, not both) if plat.startswith("macosx_") and plat.endswith("_universal2"): want = "x86_64" if platform.machine() == "x86_64" else "arm64" plat = re.sub(r"_universal2$", f"_{want}", plat) return python, abi, plat from setuptools.command.install import install class install_platlib(install): def finalize_options(self): super().finalize_options() self.install_lib = self.install_platlib here = os.path.abspath(os.path.dirname(__file__)) # https://packaging.python.org/guides/single-sourcing-package-version/ def read(*parts): with open(os.path.join(here, *parts), 'r') as fp: return fp.read() def find_version(*file_paths): version_file = read(*file_paths) version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file, re.M) if version_match: return version_match.group(1) raise RuntimeError("Unable to find version string.") version = find_version('kaldi_active_grammar', '__init__.py') if version.endswith('dev0'): version = version[:-1] + datetime.datetime.now().strftime('%Y%m%d%H%M%S') # Set branch for Kaldi source repository (maybe we should use commits instead?) if not os.environ.get('KALDI_BRANCH'): os.environ['KALDI_BRANCH'] = ('kag-v' + version) if ('dev' not in version) else 'origin/master' # Get the long description from the README file with open(os.path.join(here, 'README.md'), encoding='utf-8') as f: long_description = f.read() # Arguments marked as "Required" below must be included for upload to PyPI. # Fields marked as "Optional" may be commented out. setup( cmdclass={ 'bdist_wheel': bdist_wheel_impure, 'install': install_platlib, }, # This is the name of your project. The first time you publish this # package, this name will be registered for you. It will determine how # users can install this project, e.g.: # # $ pip install sampleproject # # And where it will live on PyPI: https://pypi.org/project/sampleproject/ # # There are some restrictions on what makes a valid project name # specification here: # https://packaging.python.org/specifications/core-metadata/#name name='kaldi-active-grammar', # Required # Versions should comply with PEP 440: # https://www.python.org/dev/peps/pep-0440/ # # For a discussion on single-sourcing the version across setup.py and the # project code, see # https://packaging.python.org/en/latest/single_source_version.html # version='0.2.0', # Required # version=open('VERSION').read().strip(), version=version, # This is a one-line description or tagline of what your project does. This # corresponds to the "Summary" metadata field: # https://packaging.python.org/specifications/core-metadata/#summary description='Kaldi speech recognition with grammars that can be set active/inactive dynamically at decode-time', # Optional # This is an optional longer description of your project that represents # the body of text which users will see when they visit PyPI. # # Often, this is the same as your README, so you can just read it in from # that file directly (as we have already done above) # # This field corresponds to the "Description" metadata field: # https://packaging.python.org/specifications/core-metadata/#description-optional long_description=long_description, # Optional # Denotes that our long_description is in Markdown; valid values are # text/plain, text/x-rst, and text/markdown # # Optional if long_description is written in reStructuredText (rst) but # required for plain-text or Markdown; if unspecified, "applications should # attempt to render [the long_description] as text/x-rst; charset=UTF-8 and # fall back to text/plain if it is not valid rst" (see link below) # # This field corresponds to the "Description-Content-Type" metadata field: # https://packaging.python.org/specifications/core-metadata/#description-content-type-optional long_description_content_type='text/markdown', # Optional (see note above) # This should be a valid link to your project's main homepage. # # This field corresponds to the "Home-Page" metadata field: # https://packaging.python.org/specifications/core-metadata/#home-page-optional url='https://github.com/daanzu/kaldi-active-grammar', # Optional # This should be your name or the name of the organization which owns the # project. author='David Zurow', # Optional # This should be a valid email address corresponding to the author listed # above. author_email='daanzu@gmail.com', # Optional license='AGPL-3.0-or-later', # Classifiers help users find your project by categorizing it. # # For a list of valid classifiers, see https://pypi.org/classifiers/ classifiers=[ # Optional # How mature is this project? Common values are # 3 - Alpha # 4 - Beta # 5 - Production/Stable 'Development Status :: 5 - Production/Stable', # Indicate who your project is intended for 'Intended Audience :: Developers', # 'Topic :: Software Development :: Build Tools', # Specify the Python versions you support here. In particular, ensure # that you indicate whether you support Python 2, Python 3 or both. # These classifiers are *not* checked by 'pip install'. See instead # 'python_requires' below. 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.9', 'Programming Language :: Python :: 3.10', 'Programming Language :: Python :: 3.11', 'Programming Language :: Python :: 3.12', 'Programming Language :: Python :: 3.13', 'Programming Language :: Python :: 3.14', 'Programming Language :: Python :: Implementation :: CPython', ], # This field adds keywords for your project which will appear on the # project page. What does your project relate to? # # Note that this is a string of words separated by whitespace, not a list. keywords='kaldi speech recognition grammar dragonfly', # Optional # You can just specify package directories manually here if your project is # simple. Or you can use find_packages(). # # Alternatively, if you just want to distribute a single Python file, use # the `py_modules` argument instead as follows, which will expect a file # called `my_module.py` to exist: # # py_modules=["my_module"], # packages=find_packages(exclude=['contrib', 'docs', 'tests']), # Required # Specify which Python versions you support. In contrast to the # 'Programming Language' classifiers above, 'pip install' will check this # and refuse to install the project if the version does not match. If you # do not support Python 2, you can simplify this to '>=3.5' or similar, see # https://packaging.python.org/guides/distributing-packages-using-setuptools/#python-requires python_requires='>=3.6, <4', # NOTE: Allowing earlier unsupported versions, even if not tested, unless we know they break # This field lists other packages that your project depends on to run. # Any package you put here will be installed by pip when your project is # installed, so they must be valid existing projects. # # For an analysis of "install_requires" vs pip's requirements files see: # https://packaging.python.org/en/latest/requirements.html install_requires=[ 'cffi >= 1.12', 'numpy >= 1.16, != 1.19.4', 'ush >= 3.1', 'six', 'futures; python_version == "2.7"', ], # Optional # List additional groups of dependencies here (e.g. development # dependencies). Users will be able to install these using the "extras" # syntax, for example: # # $ pip install sampleproject[dev] # # Similar to `install_requires` above, these must be valid existing # projects. extras_require={ # Optional 'g2p_en': ['g2p_en >= 2.1.0'], 'online': ['requests >= 2.18'], # 'dev': ['check-manifest'], # "test": [ # # See requirements-test.txt # ] }, # package_dir={ # 'kaldi_active_grammar': 'package' # }, # If there are data files included in your packages that need to be # installed, specify them here. # # If using Python 2.6 or earlier, then these have to be included in # MANIFEST.in as well. package_data={ # Optional 'kaldi_active_grammar': ['exec/*/*', 'exec/*/*/*'], '': ['LICENSE.txt'], }, # Although 'package_data' is the preferred approach, in some case you may # need to place data files outside of your packages. See: # http://docs.python.org/3.4/distutils/setupscript.html#installing-additional-files # # In this case, 'data_file' will be installed into '/my_data' # data_files=[('my_data', ['data/data_file'])], # Optional # data_files=[('my_data', ['exec/windows/dragonfly.dll'])], # Optional # data_files=[('', ['LICENSE.txt'])], # To provide executable scripts, use entry points in preference to the # "scripts" keyword. Entry points provide cross-platform support and allow # `pip` to create the appropriate form of executable for the target # platform. # # For example, the following would provide a command called `sample` which # executes the function `main` from this package when invoked: # entry_points={ # Optional # 'console_scripts': [ # 'sample=sample:main', # ], # }, # List additional URLs that are relevant to your project as a dict. # # This field corresponds to the "Project-URL" metadata fields: # https://packaging.python.org/specifications/core-metadata/#project-url-multiple-use # # Examples listed include a pattern for specifying where the package tracks # issues, where the source is hosted, where to say thanks to the package # maintainers, and where to support the project financially. The key is # what's used to render the link text on PyPI. project_urls={ # Optional 'Bug Reports': 'https://github.com/daanzu/kaldi-active-grammar/issues', 'Funding': 'https://github.com/sponsors/daanzu', # 'Say Thanks!': 'http://saythanks.io/to/example', 'Source': 'https://github.com/daanzu/kaldi-active-grammar/', }, ) ================================================ FILE: tests/conftest.py ================================================ import os from pathlib import Path import piper import pytest @pytest.fixture def change_to_test_dir(monkeypatch): monkeypatch.chdir(Path(__file__).parent) # Where model is def get_piper_model_path(): """ Get Piper model path from environment or use default. """ model_name = os.environ.get('PIPER_MODEL', 'en_US-ryan-low.onnx') model_path = Path(__file__).parent / model_name if not model_path.is_file(): raise FileNotFoundError(f"Piper model file '{model_path}' not found.") # from piper.download_voices import download_voice # download_voice(model_name, model_path.parent) return model_path @pytest.fixture(scope="session") def piper_voice(): """ Load Piper TTS voice model once per test session. """ piper_model_path = get_piper_model_path() return piper.PiperVoice.load(piper_model_path) @pytest.fixture def audio_generator(piper_voice): """ Generate audio data from text using Piper TTS. """ def _generate_audio(text, syn_config=None): if syn_config is None: syn_config = piper.SynthesisConfig( length_scale=1.5, # Slow down noise_scale=0.0, # No audio variation, for repeatable testing noise_w_scale=0.0, # No speaking variation, for repeatable testing ) audio_chunks = [] # Chunk size is variable and determined by Piper internals for chunk in piper_voice.synthesize(text, syn_config=syn_config): audio_chunks.append(chunk.audio_int16_bytes) return b''.join(audio_chunks) return _generate_audio ================================================ FILE: tests/generate_google_tts.py ================================================ #!/usr/bin/env -S uv run --script # /// script # requires-python = ">=3.12" # dependencies = [ # "google-cloud-texttospeech", # "fire", # ] # /// from google.cloud import texttospeech # pip install google-cloud-texttospeech google-auth # from google.oauth2 import service_account # creds = service_account.Credentials.from_service_account_file("service-account.json", scopes=["https://www.googleapis.com/auth/cloud-platform"]) # client = texttospeech.TextToSpeechClient(credentials=creds) # OR GOOGLE_APPLICATION_CREDENTIALS=/full/path/service-account.json client = texttospeech.TextToSpeechClient() def generate(text, out=None, voice="en-US-Studio-Q", lang="en-US", format="wav", play=False): audio_encodings = { "wav": texttospeech.AudioEncoding.LINEAR16, "mp3": texttospeech.AudioEncoding.MP3, } assert format in audio_encodings, f"Unsupported format: {format}. Supported formats: {list(audio_encodings.keys())}" if out is None: out = f"{text.replace(' ', '_')}.{format}" out = str(out) response = client.synthesize_speech( input=texttospeech.SynthesisInput(text=text), voice=texttospeech.VoiceSelectionParams(language_code=lang, name=voice), audio_config=texttospeech.AudioConfig(audio_encoding=audio_encodings[format], sample_rate_hertz=16000) ) with open(out, "wb") as f: f.write(response.audio_content) if play: import winsound winsound.PlaySound(out, winsound.SND_FILENAME) def list_voices(): for v in client.list_voices().voices: print(v.name, "-", v.language_codes) if __name__ == "__main__": import fire fire.Fire(generate) ================================================ FILE: tests/generate_piper_tts.py ================================================ #!/usr/bin/env -S uv run --script # /// script # requires-python = ">=3.12" # dependencies = [ # "piper-tts", # "kaldi-active-grammar", # "fire", # ] # /// import os import piper piper_model_path = os.path.join(os.path.dirname(__file__), 'en_US-ryan-low.onnx') voice = piper.PiperVoice.load(piper_model_path) # with wave.open("test.wav", "wb") as wav_file: # voice.synthesize_wav("Welcome to the world of speech synthesis!", wav_file) syn_config = piper.SynthesisConfig( # volume=0.5, # half as loud # length_scale=2.0, # twice as slow # noise_scale=1.0, # more audio variation # noise_w_scale=1.0, # more speaking variation length_scale=1.5, noise_scale=0.0, noise_w_scale=0.0, # normalize_audio=False, # use raw audio from voice ) # voice.synthesize_wav(..., syn_config=syn_config) text = "Welcome to the world of speech synthesis!" text = "it depends on the context" text = "up down left right" for chunk in voice.synthesize(text, syn_config=syn_config): print(chunk.sample_rate, chunk.sample_width, chunk.sample_channels) print("audio", len(chunk.audio_int16_bytes)) audio_data = chunk.audio_int16_bytes if True: from io import BytesIO import wave import winsound audio_buffer = BytesIO() with wave.open(audio_buffer, 'wb') as wav_file: wav_file.setnchannels(chunk.sample_channels) wav_file.setsampwidth(chunk.sample_width) wav_file.setframerate(chunk.sample_rate) wav_file.writeframes(chunk.audio_int16_bytes) audio_buffer.seek(0) winsound.PlaySound(audio_buffer.getvalue(), winsound.SND_MEMORY) if True: import kaldi_active_grammar as kag recognizer = kag.PlainDictationRecognizer() output_str, info = recognizer.decode_utterance(audio_data) print(repr(output_str), info) ================================================ FILE: tests/helpers.py ================================================ expected_info_keys_and_types = { 'likelihood': float, 'am_score': float, 'lm_score': float, 'confidence': float, 'expected_error_rate': float, } def assert_info_shape(info): assert isinstance(info, dict) for key, expected_type in expected_info_keys_and_types.items(): assert key in info, f"Missing key: {key}" assert isinstance(info[key], expected_type), f"Incorrect type for {key}: expected {expected_type}, got {type(info[key])}" def play_audio_on_windows(audio_bytes: bytes, sample_rate: int = 16000): """ Play raw PCM audio bytes on Windows using winsound. For interactive debugging only. """ import io import wave import winsound with io.BytesIO() as buf: with wave.open(buf, 'wb') as wf: wf.setnchannels(1) wf.setsampwidth(2) wf.setframerate(sample_rate) wf.writeframes(audio_bytes) wav_data = buf.getvalue() winsound.PlaySound(wav_data, winsound.SND_MEMORY) ================================================ FILE: tests/run_each_test_separately.py ================================================ """ Run each test in a separate process. This is the only reasonable cross-platform way to do this with pytest. """ import sys, subprocess def collect_nodeids(extra): # -q with --collect-only prints one nodeid per line r = subprocess.run( [sys.executable, "-m", "pytest", "-q", "--collect-only", *extra], capture_output=True, text=True, check=True ) return [ ln.strip().split("/")[-1] # Discard the "tests/" prefix for ln in r.stdout.splitlines() if ln.strip() and not ln.startswith(("=", "<", "[")) and not "collected in" in ln ] def main(): extra = sys.argv[1:] # e.g. ["tests", "-k", "not slow"] nodeids = collect_nodeids(extra) print(f"Collected {len(nodeids)} nodeids: {nodeids}") failed = [] for nid in nodeids: print(f"\n=== {nid} ===") rc = subprocess.call([sys.executable, "-m", "pytest", "-q", nid, *extra]) if rc != 0: failed.append(nid) print("\n========= DONE =========") if failed: print(f"\nFailures in {len(failed)} tests:") for nid in failed: print(" -", nid) sys.exit(1) else: print("\nAll tests passed.") if __name__ == "__main__": main() ================================================ FILE: tests/test_grammar.py ================================================ from typing import Callable, Optional, Union import pytest from kaldi_active_grammar import Compiler, KaldiRule, NativeWFST, WFST from tests.helpers import * class TestGrammar: @pytest.fixture(autouse=True) def setup(self, change_to_test_dir, audio_generator): self.compiler = Compiler() self.decoder = self.compiler.init_decoder() self.audio_generator = audio_generator def make_rule(self, name: str, build_func: Callable[[Union[NativeWFST, WFST]], None], **kwargs) -> KaldiRule: rule = KaldiRule(self.compiler, name, **kwargs) assert rule.name == name assert rule.fst is not None build_func(rule.fst) rule.compile() assert rule.compiled rule.load() assert rule.loaded return rule def decode(self, text_or_audio: Union[str, bytes], kaldi_rules_activity: list[bool], expected_rule: Optional[KaldiRule], expected_words: Optional[list[str]] = None, expected_words_are_dictation_mask: Optional[list[bool]] = None): if isinstance(text_or_audio, str): text = text_or_audio audio_data = self.audio_generator(text) if expected_words is None: expected_words = text.split() if text else [] else: text = None audio_data = text_or_audio if expected_words is None: expected_words = [] self.decoder.decode(audio_data, True, kaldi_rules_activity) output, info = self.decoder.get_output() assert isinstance(output, str) assert len(output) > 0 or expected_words == [] assert_info_shape(info) recognized_rule, words, words_are_dictation_mask = self.compiler.parse_output(output) if expected_rule is None: assert recognized_rule is None assert words == [] assert words_are_dictation_mask == [] else: assert recognized_rule == expected_rule assert words == expected_words if expected_words_are_dictation_mask is None: expected_words_are_dictation_mask = [False] * len(words) assert words_are_dictation_mask == expected_words_are_dictation_mask def test_simple_rule(self): def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hello') rule = self.make_rule('TestRule', _build) self.decode("hello", [True], rule) def test_epsilon_transition(self): """Test epsilon transitions between states.""" def _build(fst): initial_state = fst.add_state(initial=True) middle_state = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, middle_state, None) # epsilon transition fst.add_arc(middle_state, final_state, 'world') rule = self.make_rule('EpsilonRule', _build) self.decode("world", [True], rule) def test_multiple_paths(self): """Test grammar with multiple alternative paths (choice).""" def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) # Create three alternative paths fst.add_arc(initial_state, final_state, 'hello') fst.add_arc(initial_state, final_state, 'hi') fst.add_arc(initial_state, final_state, 'greetings') rule = self.make_rule('MultiPathRule', _build) # Test each alternative path self.decode("hello", [True], rule) def test_multiple_paths_hi(self): """Test second alternative in multiple path grammar.""" def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hello') fst.add_arc(initial_state, final_state, 'hi') fst.add_arc(initial_state, final_state, 'greetings') rule = self.make_rule('MultiPathRule2', _build) self.decode("hi", [True], rule) def test_sequential_chain(self): """Test long sequential chain of states.""" def _build(fst): initial_state = fst.add_state(initial=True) state1 = fst.add_state() state2 = fst.add_state() state3 = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, state1, 'the') fst.add_arc(state1, state2, 'quick') fst.add_arc(state2, state3, 'brown') fst.add_arc(state3, final_state, 'fox') rule = self.make_rule('SequentialRule', _build) self.decode("the quick brown fox", [True], rule) def test_diamond_pattern(self): """Test diamond pattern with branch and merge.""" def _build(fst): initial_state = fst.add_state(initial=True) branch1 = fst.add_state() branch2 = fst.add_state() merge_state = fst.add_state() final_state = fst.add_state(final=True) # Initial arc fst.add_arc(initial_state, branch1, 'start') # Two branches with different paths fst.add_arc(branch1, merge_state, 'left') fst.add_arc(branch1, branch2, 'right') fst.add_arc(branch2, merge_state, 'path') # Merge and continue fst.add_arc(merge_state, final_state, 'end') rule = self.make_rule('DiamondRule', _build) self.decode("start left end", [True], rule) def test_diamond_pattern_alt(self): """Test alternative path through diamond pattern.""" def _build(fst): initial_state = fst.add_state(initial=True) branch1 = fst.add_state() branch2 = fst.add_state() merge_state = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, branch1, 'start') fst.add_arc(branch1, merge_state, 'left') fst.add_arc(branch1, branch2, 'right') fst.add_arc(branch2, merge_state, 'path') fst.add_arc(merge_state, final_state, 'end') rule = self.make_rule('DiamondRule2', _build) self.decode("start right path end", [True], rule) def test_self_loop(self): """Test self-loop for optional repetition.""" def _build(fst): initial_state = fst.add_state(initial=True) loop_state = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, loop_state, 'repeat') fst.add_arc(loop_state, loop_state, 'again') # Self-loop fst.add_arc(loop_state, final_state, 'done') rule = self.make_rule('LoopRule', _build) self.decode("repeat again again done", [True], rule) def test_optional_path_with_epsilon(self): """Test optional path using epsilon transition.""" def _build(fst): initial_state = fst.add_state(initial=True) optional_state = fst.add_state() final_state = fst.add_state(final=True) # Direct path (skipping optional) fst.add_arc(initial_state, final_state, 'hello') # Optional path with epsilon fst.add_arc(initial_state, optional_state, None) # epsilon fst.add_arc(optional_state, final_state, 'optional') rule = self.make_rule('OptionalRule', _build) self.decode("hello", [True], rule) def test_complex_branching(self): """Test complex branching structure with multiple decision points.""" def _build(fst): initial_state = fst.add_state(initial=True) branch_a = fst.add_state() branch_b = fst.add_state() sub_branch_a1 = fst.add_state() sub_branch_a2 = fst.add_state() final_state = fst.add_state(final=True) # First level branching fst.add_arc(initial_state, branch_a, 'go') fst.add_arc(initial_state, branch_b, 'move') # Branch A has sub-branches fst.add_arc(branch_a, sub_branch_a1, 'left') fst.add_arc(branch_a, sub_branch_a2, 'right') fst.add_arc(sub_branch_a1, final_state, 'side') fst.add_arc(sub_branch_a2, final_state, 'side') # Branch B goes directly to final fst.add_arc(branch_b, final_state, 'forward') rule = self.make_rule('ComplexBranchRule', _build) self.decode("go left side", [True], rule) def test_complex_branching_alt1(self): """Test alternative path in complex branching.""" def _build(fst): initial_state = fst.add_state(initial=True) branch_a = fst.add_state() branch_b = fst.add_state() sub_branch_a1 = fst.add_state() sub_branch_a2 = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, branch_a, 'go') fst.add_arc(initial_state, branch_b, 'move') fst.add_arc(branch_a, sub_branch_a1, 'left') fst.add_arc(branch_a, sub_branch_a2, 'right') fst.add_arc(sub_branch_a1, final_state, 'side') fst.add_arc(sub_branch_a2, final_state, 'side') fst.add_arc(branch_b, final_state, 'forward') rule = self.make_rule('ComplexBranchRule2', _build) self.decode("go right side", [True], rule) def test_complex_branching_alt2(self): """Test third alternative path in complex branching.""" def _build(fst): initial_state = fst.add_state(initial=True) branch_a = fst.add_state() branch_b = fst.add_state() sub_branch_a1 = fst.add_state() sub_branch_a2 = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, branch_a, 'go') fst.add_arc(initial_state, branch_b, 'move') fst.add_arc(branch_a, sub_branch_a1, 'left') fst.add_arc(branch_a, sub_branch_a2, 'right') fst.add_arc(sub_branch_a1, final_state, 'side') fst.add_arc(sub_branch_a2, final_state, 'side') fst.add_arc(branch_b, final_state, 'forward') rule = self.make_rule('ComplexBranchRule3', _build) self.decode("move forward", [True], rule) def test_multiple_epsilon_transitions(self): """Test multiple consecutive epsilon transitions.""" def _build(fst): initial_state = fst.add_state(initial=True) eps1 = fst.add_state() eps2 = fst.add_state() eps3 = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, eps1, None) # epsilon fst.add_arc(eps1, eps2, None) # epsilon fst.add_arc(eps2, eps3, None) # epsilon fst.add_arc(eps3, final_state, 'finish') rule = self.make_rule('MultiEpsilonRule', _build) self.decode("finish", [True], rule) def test_weighted_alternatives(self): """Test weighted alternative paths (higher weight should be preferred).""" def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) # Add alternatives with different weights fst.add_arc(initial_state, final_state, 'test', weight=0.9) # Higher probability fst.add_arc(initial_state, final_state, 'test', weight=0.1) # Lower probability rule = self.make_rule('WeightedRule', _build) self.decode("test", [True], rule) def test_parallel_sequences(self): """Test parallel sequences that merge at the end.""" def _build(fst): initial_state = fst.add_state(initial=True) seq1_s1 = fst.add_state() seq1_s2 = fst.add_state() seq2_s1 = fst.add_state() final_state = fst.add_state(final=True) # Sequence 1: long path fst.add_arc(initial_state, seq1_s1, 'long') fst.add_arc(seq1_s1, seq1_s2, 'path') fst.add_arc(seq1_s2, final_state, 'here') # Sequence 2: short path fst.add_arc(initial_state, seq2_s1, 'short') fst.add_arc(seq2_s1, final_state, 'way') rule = self.make_rule('ParallelRule', _build) self.decode("long path here", [True], rule) def test_parallel_sequences_alt(self): """Test alternative parallel sequence.""" def _build(fst): initial_state = fst.add_state(initial=True) seq1_s1 = fst.add_state() seq1_s2 = fst.add_state() seq2_s1 = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, seq1_s1, 'long') fst.add_arc(seq1_s1, seq1_s2, 'path') fst.add_arc(seq1_s2, final_state, 'here') fst.add_arc(initial_state, seq2_s1, 'short') fst.add_arc(seq2_s1, final_state, 'way') rule = self.make_rule('ParallelRule2', _build) self.decode("short way", [True], rule) def test_nested_loops(self): """Test nested loop structures.""" def _build(fst): initial_state = fst.add_state(initial=True) outer_loop = fst.add_state() inner_loop = fst.add_state() exit_state = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, outer_loop, 'start') fst.add_arc(outer_loop, inner_loop, 'inner') fst.add_arc(inner_loop, inner_loop, 'repeat') # Inner self-loop fst.add_arc(inner_loop, outer_loop, 'outer') # Back to outer fst.add_arc(outer_loop, exit_state, 'exit') fst.add_arc(exit_state, final_state, 'done') rule = self.make_rule('NestedLoopRule', _build) self.decode("start inner repeat outer exit done", [True], rule) def test_multiple_entry_points(self): """Test graph with multiple entry points via epsilon.""" def _build(fst): initial_state = fst.add_state(initial=True) entry1 = fst.add_state() entry2 = fst.add_state() merge = fst.add_state() final_state = fst.add_state(final=True) # Multiple epsilon transitions to different entry points fst.add_arc(initial_state, entry1, None) # epsilon to entry1 fst.add_arc(initial_state, entry2, None) # epsilon to entry2 # Each entry has its own word fst.add_arc(entry1, merge, 'alpha') fst.add_arc(entry2, merge, 'beta') # Merge to final fst.add_arc(merge, final_state, 'end') rule = self.make_rule('MultiEntryRule', _build) self.decode("alpha end", [True], rule) def test_cascade_pattern(self): """Test cascading pattern with multiple stages.""" def _build(fst): initial_state = fst.add_state(initial=True) stage1 = fst.add_state() stage2 = fst.add_state() stage3 = fst.add_state() final_state = fst.add_state(final=True) # Stage 1: two options fst.add_arc(initial_state, stage1, 'one') fst.add_arc(initial_state, stage1, 'two') # Stage 2: connects to stage1 fst.add_arc(stage1, stage2, 'and') # Stage 3: two options from stage2 fst.add_arc(stage2, stage3, 'three') fst.add_arc(stage2, stage3, 'four') # Final fst.add_arc(stage3, final_state, 'done') rule = self.make_rule('CascadeRule', _build) self.decode("one and three done", [True], rule) def test_backtracking_pattern(self): """Test pattern that requires backtracking in search.""" def _build(fst): initial_state = fst.add_state(initial=True) trap = fst.add_state() # Dead end good_path = fst.add_state() final_state = fst.add_state(final=True) # First arc is ambiguous fst.add_arc(initial_state, trap, 'start') fst.add_arc(initial_state, good_path, 'start') # Trap has no continuation matching our test fst.add_arc(trap, trap, 'wrong') # Good path continues fst.add_arc(good_path, final_state, 'right') rule = self.make_rule('BacktrackRule', _build) self.decode("start right", [True], rule) def test_very_long_sequence(self): """Test very long sequential chain to stress test.""" def _build(fst): words = ['one', 'two', 'three', 'four', 'five', 'six', 'seven', 'eight', 'nine', 'ten'] states = [fst.add_state(initial=(i == 0), final=(i == len(words))) for i in range(len(words) + 1)] for i, word in enumerate(words): fst.add_arc(states[i], states[i + 1], word) rule = self.make_rule('LongSequenceRule', _build) self.decode("one two three four five six seven eight nine ten", [True], rule) def test_hub_and_spoke(self): """Test hub-and-spoke pattern with central node.""" def _build(fst): initial_state = fst.add_state(initial=True) hub = fst.add_state() spoke1 = fst.add_state() spoke2 = fst.add_state() spoke3 = fst.add_state() final_state = fst.add_state(final=True) # All paths go through hub fst.add_arc(initial_state, hub, 'center') # Spokes from hub fst.add_arc(hub, spoke1, 'north') fst.add_arc(hub, spoke2, 'south') fst.add_arc(hub, spoke3, 'east') # All spokes lead to final fst.add_arc(spoke1, final_state, 'end') fst.add_arc(spoke2, final_state, 'end') fst.add_arc(spoke3, final_state, 'end') rule = self.make_rule('HubSpokeRule', _build) self.decode("center north end", [True], rule) @pytest.mark.parametrize('dictation_words,expected_mask', [ ("", [False]), ("hello", [False, True]), ("hello world", [False, True, True]), ], ids=['zero_words', 'one_word', 'two_words']) def test_rule_with_dictation(self, dictation_words, expected_mask): """Test rule with dictation element: 'dictate ' with varying dictation content.""" def _build(fst): initial_state = fst.add_state(initial=True) write_state = fst.add_state() dictation_state = fst.add_state() end_state = fst.add_state() final_state = fst.add_state(final=True) fst.add_arc(initial_state, write_state, 'dictate') fst.add_arc(write_state, dictation_state, '#nonterm:dictation') fst.add_arc(dictation_state, end_state, None, '#nonterm:end') fst.add_arc(end_state, final_state, None) rule = self.make_rule('DictationRule', _build, has_dictation=True) text = f"dictate {dictation_words}".strip() self.decode(text, [True], rule, expected_words_are_dictation_mask=expected_mask) def test_no_rules(self): """Test decoding when no rules are defined.""" self.decode("hello", [], None) def test_no_active_rules(self): """Test decoding when no rules are active.""" def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hello') rule = self.make_rule('InactiveRule', _build) self.decode("hello", [False], None) def test_garbage_audio(self): """Test decoder with random noise/garbage audio.""" import random def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hello') rule = self.make_rule('NoiseRule', _build) random.seed(42) audio_data = bytes(random.randint(0, 255) for _ in range(32768)) self.decode(audio_data, [True], None) def test_empty_audio(self): """Test decoder with empty audio data.""" def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hello') rule = self.make_rule('EmptyAudioRule', _build) self.decode(b'', [True], None) def test_very_short_audio(self): """Test decoder with very short utterance.""" def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hi') rule = self.make_rule('ShortRule', _build) self.decode("hi", [True], rule) def test_multiple_utterances_sequence(self): """Test decoding multiple utterances in sequence.""" def _build(fst): initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hello') fst.add_arc(initial_state, final_state, 'world') fst.add_arc(initial_state, final_state, 'test') rule = self.make_rule('MultiUtteranceRule', _build) for text in ['hello', 'world', 'test']: self.decode(text, [True], rule) class TestAlternativeDictation: """Tests for alternative dictation feature.""" @pytest.fixture(autouse=True) def setup(self, change_to_test_dir, audio_generator): self.audio_generator = audio_generator self.alternative_dictation_calls = [] @pytest.fixture def compiler_with_mock(self): """Fixture providing compiler with mock alternative dictation.""" def mock_alternative_dictation_func(audio_data): self.alternative_dictation_calls.append(audio_data) return 'ALTERNATIVE_TEXT' return Compiler(alternative_dictation=mock_alternative_dictation_func) def create_mock_rule(self, compiler, has_dictation=True): """Helper to create a mock KaldiRule for testing.""" return KaldiRule(compiler, 'mock_rule', has_dictation=has_dictation) def parse_with_dictation_info(self, compiler, output_text, audio_data, word_align): """Helper to parse output with dictation info.""" def dictation_info_func(): return audio_data, word_align return compiler.parse_output(output_text, dictation_info_func=dictation_info_func) def test_alternative_dictation_callable_check(self): """Test that alternative_dictation must be callable.""" compiler = Compiler(alternative_dictation=lambda x: 'text') assert compiler.alternative_dictation is not None compiler = Compiler(alternative_dictation=None) assert compiler.alternative_dictation is None def test_alternative_dictation_not_called_without_dictation(self, compiler_with_mock): """Test alternative dictation is not called for rules without dictation.""" decoder = compiler_with_mock.init_decoder() rule = KaldiRule(compiler_with_mock, 'no_dictation_rule', has_dictation=False) fst = rule.fst initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'hello') rule.compile().load() decoder.decode(self.audio_generator('hello'), True, [True]) output, info = decoder.get_output() kaldi_rule, words, words_are_dictation_mask = compiler_with_mock.parse_output(output, dictation_info_func=None) assert len(self.alternative_dictation_calls) == 0 assert kaldi_rule == rule assert words == ['hello'] def test_alternative_dictation_integration_full_decode(self, change_to_test_dir): """Full integration test: rule with dictation, decode audio, alternative dictation called and replaces text.""" from kaldi_active_grammar import PlainDictationRecognizer alternative_calls = [] alternative_audio_received = [] alternative_recognized_texts = [] def alternative_dictation_func(audio_data): """Uses an independent PlainDictationRecognizer to decode the audio.""" alternative_calls.append(True) alternative_audio_received.append(len(audio_data)) alt_recognizer = PlainDictationRecognizer() alt_text, alt_info = alt_recognizer.decode_utterance(audio_data) alternative_recognized_texts.append(alt_text) return alt_text compiler = Compiler(alternative_dictation=alternative_dictation_func) decoder = compiler.init_decoder() # Create rule with dictation: "hello " rule = KaldiRule(compiler, 'dictation_rule', has_dictation=True) fst = rule.fst initial_state = fst.add_state(initial=True) hello_state = fst.add_state() dictation_state = fst.add_state() end_state = fst.add_state() final_state = fst.add_state(final=True) # Pattern: "hello" followed by dictation fst.add_arc(initial_state, hello_state, 'hello') fst.add_arc(hello_state, dictation_state, '#nonterm:dictation', '#nonterm:dictation_cloud') # #nonterm:dictation must be on ilabel; cloud variant on olabel fst.add_arc(dictation_state, end_state, None, '#nonterm:end') fst.add_arc(end_state, final_state, None) rule.compile().load() # Generate audio for "hello world" audio_data = self.audio_generator('hello world') # Decode decoder.decode(audio_data, True, [True]) output, info = decoder.get_output() # Get word alignment for alternative dictation word_align = decoder.get_word_align(output) # Create dictation_info_func that returns audio and word_align def dictation_info_func(): return audio_data, word_align # Parse with alternative dictation kaldi_rule, words, words_are_dictation_mask = compiler.parse_output( output, dictation_info_func=dictation_info_func) # Verify alternative dictation was called assert len(alternative_calls) > 0, "Alternative dictation should have been called" assert len(alternative_audio_received) > 0, "Alternative dictation should have received audio" assert len(alternative_recognized_texts) > 0, "Alternative dictation should have recognized text" # Verify the alternative recognizer produced some output alt_text = alternative_recognized_texts[0] assert alt_text, f"Alternative recognizer should produce text, got: {alt_text}" # The alternative text should be in the final words (replacing original dictation) words_str = ' '.join(words) assert alt_text in words_str or any(word in words for word in alt_text.split()), \ f"Alternative text '{alt_text}' should be in words: {words}" # Verify 'hello' is still there (not part of dictation) assert 'hello' in words, f"Hello word should be preserved: {words}" # Verify rule was recognized assert kaldi_rule == rule # Verify dictation mask is correct assert len(words) == len(words_are_dictation_mask) assert words_are_dictation_mask[words.index('hello')] == False, "Hello should not be marked as dictation" def test_alternative_dictation_not_called_without_cloud_nonterm(self, compiler_with_mock): """Test alternative dictation not called when #nonterm:dictation_cloud not in output.""" decoder = compiler_with_mock.init_decoder() rule = KaldiRule(compiler_with_mock, 'no_cloud_rule', has_dictation=True) fst = rule.fst initial_state = fst.add_state(initial=True) final_state = fst.add_state(final=True) fst.add_arc(initial_state, final_state, 'test') rule.compile().load() decoder.decode(self.audio_generator('test'), True, [True]) output, info = decoder.get_output() mock_audio = b'mock_audio_data' mock_word_align = [('test', 0, 1000)] kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler_with_mock, output, mock_audio, mock_word_align) assert len(self.alternative_dictation_calls) == 0 def test_alternative_dictation_word_align_parsing(self, compiler_with_mock): """Test parsing of word_align data for dictation spans.""" output_text = '#nonterm:rule0 start #nonterm:dictation_cloud original text #nonterm:end finish' mock_audio = b'\x00' * 32000 mock_word_align = [ ('#nonterm:rule0', 0, 0), ('start', 0, 8000), ('#nonterm:dictation_cloud', 8000, 0), ('original', 8000, 4000), ('text', 12000, 4000), ('#nonterm:end', 16000, 0), ('finish', 16000, 8000), ] self.create_mock_rule(compiler_with_mock) kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler_with_mock, output_text, mock_audio, mock_word_align) assert len(self.alternative_dictation_calls) == 1 assert len(self.alternative_dictation_calls[0]) > 0 assert 'ALTERNATIVE_TEXT' in words or words == ['start', 'finish'] @pytest.mark.parametrize('output_text,word_align,audio_size,expected_audio_size', [ ( '#nonterm:rule0 start #nonterm:dictation_cloud final words #nonterm:end', [ ('#nonterm:rule0', 0, 0), ('start', 0, 8000), ('#nonterm:dictation_cloud', 8000, 0), ('final', 8000, 4000), ('words', 12000, 4000), ('#nonterm:end', 16000, 0), ], 32000, 24000, # 32000 - 8000 ), ( '#nonterm:rule0 start #nonterm:dictation_cloud middle text #nonterm:end finish', [ ('#nonterm:rule0', 0, 0), ('start', 0, 4000), ('#nonterm:dictation_cloud', 4000, 0), ('middle', 4000, 4000), ('text', 8000, 4000), ('#nonterm:end', 12000, 0), ('finish', 16000, 4000), ], 32000, 10000, # 14000 - 4000 (half gap to next word) ), ], ids=['end_of_utterance', 'middle_of_utterance']) def test_alternative_dictation_span_calculation(self, compiler_with_mock, output_text, word_align, audio_size, expected_audio_size): """Test dictation span calculation for various positions.""" mock_audio = b'\x00' * audio_size self.create_mock_rule(compiler_with_mock) kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler_with_mock, output_text, mock_audio, word_align) assert len(self.alternative_dictation_calls) == 1 assert len(self.alternative_dictation_calls[0]) == expected_audio_size def test_alternative_dictation_multiple_spans(self): """Test handling multiple dictation spans in single utterance.""" call_count = [0] def multi_alternative_func(audio_data): call_count[0] += 1 return f'ALT_{call_count[0]}' compiler = Compiler(alternative_dictation=multi_alternative_func) output_text = '#nonterm:rule0 start #nonterm:dictation_cloud first #nonterm:end middle #nonterm:dictation_cloud second #nonterm:end finish' mock_audio = b'\x00' * 48000 mock_word_align = [ ('#nonterm:rule0', 0, 0), ('start', 0, 4000), ('#nonterm:dictation_cloud', 4000, 0), ('first', 4000, 4000), ('#nonterm:end', 8000, 0), ('middle', 12000, 4000), ('#nonterm:dictation_cloud', 16000, 0), ('second', 16000, 4000), ('#nonterm:end', 20000, 0), ('finish', 24000, 4000), ] self.create_mock_rule(compiler) kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler, output_text, mock_audio, mock_word_align) assert call_count[0] == 2 @pytest.mark.parametrize('alternative_func,expected_words', [ (lambda x: None, ['original', 'text']), (lambda x: '', ['original', 'text']), ], ids=['returns_none', 'returns_empty_string']) def test_alternative_dictation_fallback(self, alternative_func, expected_words): """Test fallback to original text when alternative_dictation returns falsy value.""" compiler = Compiler(alternative_dictation=alternative_func) output_text = '#nonterm:rule0 #nonterm:dictation_cloud original text #nonterm:end' mock_audio = b'\x00' * 16000 mock_word_align = [ ('#nonterm:rule0', 0, 0), ('#nonterm:dictation_cloud', 0, 0), ('original', 0, 4000), ('text', 4000, 4000), ('#nonterm:end', 8000, 0), ] self.create_mock_rule(compiler) kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler, output_text, mock_audio, mock_word_align) for expected_word in expected_words: assert expected_word in words def test_alternative_dictation_exception_handling(self): """Test that exceptions in alternative_dictation are caught and logged.""" def failing_func(audio_data): raise ValueError('Test exception') compiler = Compiler(alternative_dictation=failing_func) output_text = '#nonterm:rule0 #nonterm:dictation_cloud original #nonterm:end' mock_audio = b'\x00' * 8000 mock_word_align = [ ('#nonterm:rule0', 0, 0), ('#nonterm:dictation_cloud', 0, 0), ('original', 0, 4000), ('#nonterm:end', 4000, 0), ] self.create_mock_rule(compiler) kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler, output_text, mock_audio, mock_word_align) assert 'original' in words def test_alternative_dictation_invalid_type_raises(self): """Test that invalid alternative_dictation type raises TypeError.""" compiler = Compiler(alternative_dictation='not_callable') output_text = '#nonterm:rule0 #nonterm:dictation_cloud text #nonterm:end' mock_audio = b'\x00' * 8000 mock_word_align = [ ('#nonterm:rule0', 0, 0), ('#nonterm:dictation_cloud', 0, 0), ('text', 0, 4000), ('#nonterm:end', 4000, 0), ] self.create_mock_rule(compiler) kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler, output_text, mock_audio, mock_word_align) assert words is not None def test_alternative_dictation_audio_slice_accuracy(self): """Test that correct audio slice is passed to alternative_dictation.""" received_audio = [] def capture_audio_func(audio_data): received_audio.append(audio_data) return 'replaced' compiler = Compiler(alternative_dictation=capture_audio_func) output_text = '#nonterm:rule0 #nonterm:dictation_cloud test #nonterm:end' mock_audio = b'\x01' * 4000 + b'\x02' * 4000 + b'\x03' * 4000 mock_word_align = [ ('#nonterm:rule0', 0, 0), ('#nonterm:dictation_cloud', 4000, 0), ('test', 4000, 4000), ('#nonterm:end', 8000, 0), ] self.create_mock_rule(compiler) kaldi_rule, words, words_are_dictation_mask = self.parse_with_dictation_info(compiler, output_text, mock_audio, mock_word_align) assert len(received_audio) == 1 assert received_audio[0] == b'\x02' * 4000 + b'\x03' * 4000 ================================================ FILE: tests/test_package.py ================================================ import re def test_import_and_version(): import kaldi_active_grammar as kag assert isinstance(kag.__version__, str) assert kag.__version__.strip() != "" version_pattern = r'^\d+\.\d+\.\d+(?:[-+].+)?$' assert re.match(version_pattern, kag.__version__), f"Version '{kag.__version__}' does not match semantic versioning format" ================================================ FILE: tests/test_plain_dictation.py ================================================ import math import random import pytest from kaldi_active_grammar import PlainDictationRecognizer from tests.helpers import * @pytest.fixture def recognizer(change_to_test_dir): return PlainDictationRecognizer() def test_initialization(recognizer): assert isinstance(recognizer, PlainDictationRecognizer) assert hasattr(recognizer, 'decoder') assert hasattr(recognizer, '_compiler') @pytest.mark.parametrize("test_text", [ "hello world", "this is a longer sentence to test the speech recognition capabilities", "testing active grammar framework", "hello there", "one two three four five", "testing numbers and words", ]) def test_basic_dictation(recognizer, audio_generator, test_text): audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert output_str == test_text assert_info_shape(info) def test_empty_audio(recognizer): output_str, info = recognizer.decode_utterance(b'') assert isinstance(output_str, str) assert output_str == "" assert_info_shape(info) def test_garbage_audio(recognizer): random.seed(42) audio_data = bytes(random.randint(0, 255) for _ in range(32768)) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert output_str == "" assert_info_shape(info) def test_multiple_utterances(recognizer, audio_generator): test_utterances = [ "first utterance", "second utterance here", "and a third one", ] for test_text in test_utterances: audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert output_str == test_text assert_info_shape(info) class TestPlainDictationWithFST: """Test PlainDictationRecognizer using HCLG.fst file""" @pytest.fixture(autouse=True) def setup(self, change_to_test_dir): try: self.recognizer = PlainDictationRecognizer(fst_file='HCLG.fst') self.has_hclg = True except Exception: self.has_hclg = False pytest.skip("HCLG.fst not available for testing") def test_initialization(self): if not self.has_hclg: pytest.skip("HCLG.fst not available") assert isinstance(self.recognizer, PlainDictationRecognizer) assert hasattr(self.recognizer, 'decoder') assert hasattr(self.recognizer, '_model') def test_basic_dictation(self, audio_generator): if not self.has_hclg: pytest.skip("HCLG.fst not available") test_text = "testing plain decoder" audio_data = audio_generator(test_text) output_str, info = self.recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert_info_shape(info) @pytest.mark.parametrize("chunk_size", [512, 1024, 2048, 4096, 8192, 16384]) @pytest.mark.parametrize("test_text", [ "testing small chunk size", "medium chunk size testing", "large chunk size for testing", ]) def test_chunked_decode(recognizer, audio_generator, chunk_size, test_text): audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data, chunk_size=chunk_size) assert isinstance(output_str, str) assert output_str == test_text assert_info_shape(info) def test_custom_tmp_dir(change_to_test_dir, audio_generator, tmp_path): recognizer = PlainDictationRecognizer(tmp_dir=str(tmp_path)) test_text = "custom temporary directory" audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert output_str == test_text assert_info_shape(info) def test_custom_config(change_to_test_dir, audio_generator): config = { 'beam': 13.0, 'max_active': 7000, } recognizer = PlainDictationRecognizer(config=config) test_text = "custom configuration test" audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert output_str == test_text assert_info_shape(info) def test_very_short_audio(recognizer, audio_generator): test_text = "hi" audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert_info_shape(info) def test_very_long_audio(recognizer, audio_generator): test_text = " ".join([ "this is a very long sentence that goes on and on", "with many words to test the handling of extended audio", "and ensure that the decoder can process lengthy utterances", "without any issues or errors occurring during processing", ]) audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert_info_shape(info) def test_repeated_words(recognizer, audio_generator): test_text = "repeat repeat repeat the words" audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert isinstance(output_str, str) assert_info_shape(info) def test_sequential_empty_audio(recognizer): for _ in range(3): output_str, info = recognizer.decode_utterance(b'') assert isinstance(output_str, str) assert output_str == "" assert_info_shape(info) def test_alternating_empty_and_valid(recognizer, audio_generator): test_text = "valid audio" output_str, info = recognizer.decode_utterance(b'') assert output_str == "" assert_info_shape(info) audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert output_str == test_text assert_info_shape(info) output_str, info = recognizer.decode_utterance(b'') assert output_str == "" assert_info_shape(info) def test_info_structure(recognizer, audio_generator): test_text = "check info dictionary" audio_data = audio_generator(test_text) output_str, info = recognizer.decode_utterance(audio_data) assert_info_shape(info) assert 0.0 <= info['confidence'] <= 1.0 or math.isnan(info['confidence']) assert 0.0 <= info['expected_error_rate'] <= 1.0 or math.isnan(info['expected_error_rate']) def test_info_consistency(change_to_test_dir, audio_generator): test_text = "consistent info values" audio_data = audio_generator(test_text) recognizer1 = PlainDictationRecognizer() output_str1, info1 = recognizer1.decode_utterance(audio_data) recognizer2 = PlainDictationRecognizer() output_str2, info2 = recognizer2.decode_utterance(audio_data) assert output_str1 == output_str2 assert_info_shape(info1) assert_info_shape(info2) threshold_pct = 0.01 assert abs(info1['likelihood'] - info2['likelihood']) / abs(info1['likelihood']) < threshold_pct assert abs(info1['am_score'] - info2['am_score']) / abs(info1['am_score']) < threshold_pct assert abs(info1['lm_score'] - info2['lm_score']) / abs(info1['lm_score']) < threshold_pct