Repository: elixir-lang/gettext Branch: main Commit: 05992bb1a055 Files: 59 Total size: 298.1 KB Directory structure: gitextract_oi5hxoga/ ├── .formatter.exs ├── .github/ │ ├── CODE_OF_CONDUCT.md │ ├── CONTRIBUTING.md │ ├── SECURITY.md │ ├── dependabot.yml │ └── workflows/ │ ├── main.yml │ └── publish-to-hex.yml ├── .gitignore ├── CHANGELOG.md ├── LICENSE ├── README.md ├── coveralls.json ├── lib/ │ ├── gettext/ │ │ ├── application.ex │ │ ├── backend.ex │ │ ├── compiler.ex │ │ ├── error.ex │ │ ├── extractor.ex │ │ ├── extractor_agent.ex │ │ ├── fuzzy.ex │ │ ├── interpolation/ │ │ │ └── default.ex │ │ ├── interpolation.ex │ │ ├── macros.ex │ │ ├── merger.ex │ │ ├── missing_bindings_error.ex │ │ ├── plural.ex │ │ └── plural_form_error.ex │ ├── gettext.ex │ └── mix/ │ └── tasks/ │ ├── compile.gettext.ex │ ├── gettext.extract.ex │ └── gettext.merge.ex ├── mix.exs └── test/ ├── fixtures/ │ ├── bad_messages/ │ │ └── ru/ │ │ └── LC_MESSAGES/ │ │ └── errors.po │ ├── bom.po │ ├── empty.po │ ├── invalid_syntax_error.po │ ├── invalid_token_error.po │ ├── multi_messages/ │ │ ├── es/ │ │ │ └── LC_MESSAGES/ │ │ │ └── default.po │ │ └── it/ │ │ └── LC_MESSAGES/ │ │ ├── default.po │ │ └── errors.po │ ├── po_editors/ │ │ ├── poedit.po │ │ └── poeditor.com.po │ ├── single_messages/ │ │ ├── it/ │ │ │ └── LC_MESSAGES/ │ │ │ ├── default.po │ │ │ ├── errors.po │ │ │ └── interpolations.po │ │ └── ja/ │ │ └── LC_MESSAGES/ │ │ └── errors.po │ └── valid.po ├── gettext/ │ ├── backend_test.exs │ ├── extractor_test.exs │ ├── fuzzy_test.exs │ ├── interpolation/ │ │ └── default_test.exs │ ├── macros_test.exs │ ├── merger_test.exs │ ├── new_backend_setup_test.exs │ └── plural_test.exs ├── gettext_test.exs ├── mix/ │ └── tasks/ │ ├── gettext.extract_test.exs │ └── gettext.merge_test.exs ├── support/ │ └── mix_project_helpers.ex └── test_helper.exs ================================================ FILE CONTENTS ================================================ ================================================ FILE: .formatter.exs ================================================ # Used by "mix format" and to export configuration. [ inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"] ] ================================================ FILE: .github/CODE_OF_CONDUCT.md ================================================ # Code of Conduct This project follows the [Elixir Code of Conduct](https://github.com/elixir-lang/elixir/blob/main/CODE_OF_CONDUCT.md). We are committed to providing a friendly, safe, and welcoming environment for all contributors. By participating in this project, you agree to abide by the standards outlined in the Elixir Code of Conduct. If you experience or witness behavior that violates the Code of Conduct, please report it to: [elixir-lang-conduct@googlegroups.com](mailto:elixir-lang-conduct@googlegroups.com). Thank you for helping us foster a respectful and inclusive community! ================================================ FILE: .github/CONTRIBUTING.md ================================================ # Contributing to `gettext` ## Welcome! We look forward to your contributions! Here are some examples how you can contribute: - [Report an issue](https://github.com/elixir-gettext/gettext/issues/new) - [Send a pull request](https://github.com/elixir-gettext/gettext/pulls) ## We have a Code of Conduct Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms. ## Any contributions you make will be under the Apache 2.0 License When you submit code changes, your submissions are understood to be under the same [Apache 2.0](https://github.com/elixir-gettext/gettext/blob/main/LICENSE) that covers the project. By contributing to this project, you agree that your contributions will be licensed under its Apache 2.0 License. ## Write bug reports with detail, background, and sample code In your bug report, please provide the following: - A quick summary and/or background - Steps to reproduce - Be specific! - Give sample code if you can. - What you expected would happen - What actually happens - Notes (possibly including why you think this might be happening, or stuff you - tried that didn't work) Please post code and output as text ([using proper markup](https://guides.github.com/features/mastering-markdown/)). Do not post screenshots of code or output. ## Workflow for Pull Requests 1. Fork the repository. 2. Create your branch from `main` if you plan to implement new functionality or change existing code significantly; create your branch from the oldest branch that is affected by the bug if you plan to fix a bug. 3. Implement your change and add tests for it. 4. Ensure the test suite passes. 5. Ensure the code complies with our coding guidelines (see below). 6. Send that pull request! Please make sure you have [set up your user name and email address](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) for use with Git. Strings such as `silly nick name ` look really stupid in the commit history of a project. We encourage you to [sign your Git commits with your GPG key](https://docs.github.com/en/github/authenticating-to-github/signing-commits). Pull requests for new features must be based on the `main` branch. We are trying to keep backwards compatibility breaks in `gettext` to a minimum. Please take this into account when proposing changes. Due to time constraints, we are not always able to respond as quickly as we would like. Please do not take delays personal and feel free to remind us if you feel that we forgot to respond. ## Coding Guidelines This project comes with configuration (located in `.formatter.exs` in the repository) that you can use to (re)format your source code for compliance with this project's coding guidelines: ```bash $ mix format ``` Please understand that we will not accept a pull request when its changes violate this project's coding guidelines. ## Using `gettext` from a Git checkout The following commands can be used to perform the initial checkout of `gettext`: ```bash $ git clone git@github.com:elixir-gettext/gettext.git $ cd gettext ``` Install `gettext`'s dependencies using [mix](https://hexdocs.pm/mix/Mix.html): ```bash $ mix deps.get ``` ## Running `gettext`'s test suite After following the steps shown above, `gettext`'s test suite is run like this: ```bash $ mix test ``` ## Generating `gettext` Documentation To generate the documentation for the library, run: ```bash $ mix docs ``` ================================================ FILE: .github/SECURITY.md ================================================ # Security Policy [![OpenSSF Vulnerability Disclosure](https://img.shields.io/badge/OpenSSF-Vulnerability_Disclosure-green)][openssf-cvd-finders-guide] [![GitHub Report](https://img.shields.io/badge/GitHub-Security_Advisories-blue)][github-advisory-new] [![Email Report](https://img.shields.io/badge/Email-hi%40andrealeopardi.com-blue)][email] We take the security of this software seriously and are committed to ensuring that any vulnerabilities are addressed promptly and effectively. This repository follows the OpenSSF [Vulnerability Disclosure guide][openssf-cvd-guide]. You can learn more about it in the [Finders Guide][openssf-cvd-finders-guide]. ## Reporting Security Issues If you believe you have found a security vulnerability in this repository, please report it via [GitHub Security Vulnerability Reporting][github-advisory-new] or via email to [`hi@andrealeopardi.com`][email] if that is more suitable for you. **Please do not report vulnerabilities through public channels** such as GitHub issues, discussions, or pull requests, to avoid exposing the details of the issue before it has been properly addressed. We don't implement a bug bounty program or bounty rewards, but will work with you to ensure that your findings get the appropriate handling. When reporting a vulnerability, please include as much detail as possible to help us triage and resolve the issue efficiently. Information that will be specially helpful includes: - The type of issue (e.g., buffer overflow, SQL injection, cross-site scripting, etc.) - Full paths of source file(s) related to the issue - The location of the affected source code (e.g., tag, branch, commit, or direct URL) - Any special configuration required to reproduce the issue - Step-by-step instructions to reproduce the issue - Proof-of-concept or exploit code (if available) - The potential impact, including how the issue might be exploited by an attacker Our vulnerability management team will respond within 3 working days of your report. If the issue is confirmed as a vulnerability, we will open a Security Advisory. This project follows a 90-day disclosure timeline. If you have any questions about reporting security issues, please contact our vulnerability management team at [`hi@andrealeopardi.com`][email]. [openssf-cvd-guide]: https://github.com/ossf/oss-vulnerability-guide/tree/main [openssf-cvd-finders-guide]: https://github.com/ossf/oss-vulnerability-guide/blob/main/finder-guide.md [github-advisory-new]: /security/advisories/new [email]: mailto:hi@andrealeopardi.com ================================================ FILE: .github/dependabot.yml ================================================ version: 2 updates: - package-ecosystem: github-actions directory: "/" schedule: interval: monthly open-pull-requests-limit: 10 ================================================ FILE: .github/workflows/main.yml ================================================ name: CI on: pull_request: push: branches: - main permissions: contents: read jobs: test: name: Test (Elixir ${{ matrix.elixir }} | Erlang/OTP ${{ matrix.erlang }}) runs-on: "${{ matrix.os }}" strategy: fail-fast: false matrix: include: - os: ubuntu-22.04 erlang: "27.2" elixir: "1.18" lint: true coverage: true - os: ubuntu-22.04 erlang: "24.2" elixir: "1.14" env: MIX_ENV: test GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Install OTP and Elixir uses: erlef/setup-beam@fc68ffb90438ef2936bbb3251622353b3dcb2f93 # v1.24.0 with: otp-version: ${{ matrix.erlang }} elixir-version: ${{ matrix.elixir }} - name: Install dependencies run: mix deps.get --check-locked - name: Check no unused dependencies run: mix deps.unlock --check-unused if: ${{ matrix.lint }} - name: Check formatting run: mix format --check-formatted if: ${{ matrix.lint }} - name: Compile with --warnings-as-errors run: mix compile --warnings-as-errors if: ${{ matrix.lint }} - name: Run tests run: mix test if: ${{ !matrix.coverage }} - name: Run tests with code coverage run: mix coveralls.github if: ${{ matrix.coverage }} ================================================ FILE: .github/workflows/publish-to-hex.yml ================================================ name: Publish to Hex on: push: tags: - v* permissions: contents: read jobs: publish: name: Publish to Hex runs-on: ubuntu-latest steps: - name: Checkout this repository uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 - name: Install Erlang and Elixir uses: erlef/setup-beam@fc68ffb90438ef2936bbb3251622353b3dcb2f93 # v1.24.0 with: otp-version: "27.2" elixir-version: "1.18" - name: Fetch dependencies run: mix deps.get - name: Publish to Hex run: mix hex.publish --yes env: HEX_API_KEY: ${{ secrets.HEX_API_KEY }} ================================================ FILE: .gitignore ================================================ /_build /cover /deps /doc /src/gettext_po_parser.erl /tmp erl_crash.dump *.ez ================================================ FILE: CHANGELOG.md ================================================ # Changelog ## v1.0.2 * Only skip manifest removal on Elixir v1.19.3+ ## v1.0.1 (retired) * Remove unnecessary cleaning of Elixir manifests ## v1.0.0 This is the first 1.0 release of Gettext, a silly 10 years (and 6 months) after we started working on it. There are *very few changes* from the latest 0.26 release, and none of them are breaking. Here are the new goodies: * Add support for concatenating sigils if all parts are known at compile time (such as `"Hello " <> ~s(world)`). * Significantly increase the timeout for `mix gettext.extract` to two minutes. * Add `Gettext.put_locale!/2`. Happy 10+ years of Elixir translations everyone! 🎉 ## Previous versions [See the CHANGELOG for versions before v1.0](https://github.com/elixir-gettext/gettext/blob/v1.0.0/CHANGELOG.md). ================================================ FILE: LICENSE ================================================ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright 2015 Plataformatec Copyright 2020 Dashbit Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================ FILE: README.md ================================================ # Gettext [![hex.pm badge](https://img.shields.io/badge/Package%20on%20hex.pm-informational)](https://hex.pm/packages/gettext) [![Documentation badge](https://img.shields.io/badge/Documentation-ff69b4)][docs-gettext] ![CI badge](https://github.com/elixir-gettext/gettext/workflows/CI/badge.svg) Gettext is an **internationalization** (i18n) and **localization** (l10n) system commonly used for writing multilingual programs. Gettext is a standard for i18n in different communities, meaning there is a great set of tooling for developers and translators. This project is an implementation of the Gettext system in Elixir. ## Installation Add `:gettext` to your list of dependencies in `mix.exs` (use `$ mix hex.info gettext` to find the latest version): ```elixir defp deps do [ {:gettext, ">= 0.0.0"} ] end ``` Documentation for `Gettext` is [available on Hex][docs-gettext]. ## Usage To use Gettext, define a **Gettext backend**: ```elixir defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end ``` and invoke the Gettext API, which consists of the `*gettext` macros that get imported if you `use Gettext`: ```elixir use Gettext, backend: MyApp.Gettext # Simple message gettext("Here is one string to translate") # Plural message number_of_apples = 4 ngettext("The apple is ripe", "The apples are ripe", number_of_apples) # Domain-based message dgettext("errors", "Here is an error message to translate") ``` Messages in Gettext are stored in Portable Object files (`.po`). Such files must be placed at `priv/gettext/LOCALE/LC_MESSAGES/DOMAIN.po`, where `LOCALE` is the locale and `DOMAIN` is the domain (the default domain is called `default`). For example, the messages for `pt_BR` from the first two `*gettext` calls in the snippet above must be placed in the `priv/gettext/pt_BR/LC_MESSAGES/default.po` file with the following contents: ```pot msgid "Here is one string to translate" msgstr "Aqui está um texto para traduzir" msgid "Here is the string to translate" msgid_plural "Here are the strings to translate" msgstr[0] "Aqui está o texto para traduzir" msgstr[1] "Aqui estão os textos para traduzir" ``` `.po` files are text-based and can be edited directly by translators. Some may even use existing tools for managing them, such as [Poedit][poedit] or [poeditor.com][poeditor.com]. Finally, because messages are based on strings, your source code does not lose readability as you still see literal strings, like `gettext("here is an example")`, instead of paths like `translate("some.path.convention")`. Read the [documentation for the `Gettext` module][docs-gettext-module] for more information on locales, interpolation, pluralization, and other features. ## Workflow Gettext is able to automatically extract messages from your source code, alleviating developers and translators from the repetitive and error-prone work of maintaining message files. When extracted from source, Gettext places messages into `.pot` files, which are template files. You can then merge those templates files into message files for each specific locale your application is being currently translated to. In other words, the typical workflow looks like this: 1. Add `gettext` calls to your source code. No need to touch message files at this point as Gettext will return the given string if no message is available: ```elixir gettext("Welcome back!") ``` 2. Once changes to the source are complete, automatically sync all existing entries to `.pot` (template files) in `priv/gettext` by running: ```bash mix gettext.extract ``` 3. You can then merge `.pot` files into locale-specific `.po` files: ```bash # Merge .pot into all locales mix gettext.merge priv/gettext # Merge .pot into one specific locale mix gettext.merge priv/gettext --locale en ``` It is also possible to both extract and merge messages in one step with `mix gettext.extract --merge`. ## License Copyright 2015 Plataformatec Copyright 2020 Dashbit Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at: > Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. [docs-gettext]: http://hexdocs.pm/gettext [docs-gettext-module]: http://hexdocs.pm/gettext/Gettext.html [poedit]: http://poedit.net/ [poeditor.com]: https://poeditor.com ================================================ FILE: coveralls.json ================================================ { "coverage_options": { "treat_no_relevant_lines_as_covered": true }, "skip_files": [ "src/gettext_po_parser.erl", "lib/mix/tasks/compile.gettext.ex" ] } ================================================ FILE: lib/gettext/application.ex ================================================ defmodule Gettext.Application do @moduledoc false use Application @impl true def start(_type, _args) do children = [Gettext.ExtractorAgent] Supervisor.start_link(children, strategy: :one_for_one) end end ================================================ FILE: lib/gettext/backend.ex ================================================ defmodule Gettext.Backend do @moduledoc """ Defines a Gettext backend. ## Usage A Gettext **backend** must `use` this module. defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Using this module generates all the callbacks required by the `Gettext.Backend` behaviour into the module that uses it. For more options and information, see `Gettext`. > #### `use Gettext.Backend` Is a Recent Feature {: .info} > > Before version v0.26.0, you could only `use Gettext` to generate a backend. > > Version v0.26.0 changes the way backends work so that now a Gettext backend > must `use Gettext.Backend`, while to use the functions in the backend you > will do `use Gettext, backend: MyApp.Gettext`. """ defmacro __using__(opts) do # TODO: From Elixir v1.13 onwards, use compile_env and remove this if. env_fun = if function_exported?(Module, :attributes_in, 1), do: :compile_env, else: :get_env quote do require Logger opts = unquote(opts) otp_app = Keyword.fetch!(opts, :otp_app) @gettext_opts opts |> Keyword.merge(Application.unquote(env_fun)(otp_app, __MODULE__, [])) |> Keyword.put_new(:interpolation, Gettext.Interpolation.Default) @interpolation Keyword.fetch!(@gettext_opts, :interpolation) @before_compile Gettext.Compiler def handle_missing_bindings(exception, incomplete) do _ = Logger.error(Exception.message(exception)) incomplete end defoverridable handle_missing_bindings: 2 def handle_missing_translation(_locale, domain, _msgctxt, msgid, bindings) do Gettext.Compiler.warn_if_domain_contains_slashes(domain) with {:ok, interpolated} <- @interpolation.runtime_interpolate(msgid, bindings), do: {:default, interpolated} end def handle_missing_plural_translation( _locale, domain, _msgctxt, msgid, msgid_plural, n, bindings ) do Gettext.Compiler.warn_if_domain_contains_slashes(domain) string = if n == 1, do: msgid, else: msgid_plural bindings = Map.put(bindings, :count, n) with {:ok, interpolated} <- @interpolation.runtime_interpolate(string, bindings), do: {:default, interpolated} end defoverridable handle_missing_translation: 5, handle_missing_plural_translation: 7 end end @doc """ Default handling for missing bindings. This function is called when there are missing bindings in a message. It takes a `Gettext.MissingBindingsError` struct and the message with the wrong bindings left as is with the `%{}` syntax. For example, if something like this is called: gettext("Hello %{name}, your favorite color is %{color}", name: "Jane", color: "blue") and our `it/LC_MESSAGES/default.po` looks like this: msgid "Hello %{name}, your favorite color is %{color}" msgstr "Ciao %{name}, il tuo colore preferito è %{colour}" # (typo) then Gettext will call: MyApp.Gettext.handle_missing_bindings(exception, "Ciao Jane, il tuo colore preferito è %{colour}") where `exception` is a struct that looks like this: %Gettext.MissingBindingsError{ backend: MyApp.Gettext, domain: "default", locale: "it", msgid: "Ciao %{name}, il tuo colore preferito è %{colour}", bindings: [:colour], } The return value of the `c:handle_missing_bindings/2` callback is used as the translated string that the message macros and functions return. The default implementation for this function uses `Logger.error/1` to warn about the missing binding and returns the translated message with the incomplete bindings. This function can be overridden. For example, to raise when there are missing bindings: def handle_missing_bindings(exception, _incomplete) do raise exception end """ @callback handle_missing_bindings(Gettext.MissingBindingsError.t(), binary) :: binary | no_return @doc """ Default handling for messages with a missing message. When a Gettext function/macro is called with a string to translate into a locale but that locale doesn't provide a message for that string, this callback is invoked. `msgid` is the string that Gettext tried to translate. This function should return `{:ok, translated}` if a message can be fetched or constructed for the given string. If you cannot find a message, it should return `{:default, translated}`, where the translated string defaults to the interpolated msgid. You can, however, customize the default to, for example, pick the message from the default locale. The important is to return `:default` instead of `:ok` whenever the result does not quite match the requested locale. Earlier versions of this library provided a callback without msgctxt. Users implementing that callback will still get the same results, but they are encouraged to switch to the new 5-argument version. """ @callback handle_missing_translation( Gettext.locale(), domain :: String.t(), msgctxt :: String.t(), msgid :: String.t(), bindings :: map() ) :: {:ok, String.t()} | {:default, String.t()} | {:missing_bindings, String.t(), [atom]} @doc """ Default handling for plural messages with a missing message. Same as `c:handle_missing_translation/5`, but for plural messages. In this case, `n` is the number used for pluralizing the translated string. Earlier versions of this library provided a callback without msgctxt. Users implementing that callback will still get the same results, but they are encouraged to switch to the new 7-argument version. """ @callback handle_missing_plural_translation( Gettext.locale(), domain :: String.t(), msgctxt :: String.t(), msgid :: String.t(), msgid_plural :: String.t(), n :: non_neg_integer(), bindings :: map() ) :: {:ok, String.t()} | {:default, String.t()} | {:missing_bindings, String.t(), [atom]} @doc """ Translates a message. See `Gettext.gettext/3` for more information. """ @doc since: "0.26.0" @callback lgettext( Gettext.locale(), domain :: String.t(), msgctxt :: String.t() | nil, msgid :: String.t(), bindings :: map() ) :: {:ok, String.t()} | {:default, String.t()} | {:missing_bindings, String.t(), [atom]} @doc """ Translates a plural message. See `Gettext.ngettext/5` for more information. """ @doc since: "0.26.0" @callback lngettext( Gettext.locale(), domain :: String.t(), msgctxt :: String.t() | nil, msgid :: String.t(), msgid_plural :: String.t(), n :: non_neg_integer(), bindings :: map() ) :: {:ok, String.t()} | {:default, String.t()} | {:missing_bindings, String.t(), [atom]} end ================================================ FILE: lib/gettext/compiler.ex ================================================ defmodule Gettext.Compiler do @moduledoc false require Logger alias Expo.Message alias Expo.Messages alias Expo.PO alias Gettext.Plural @default_priv "priv/gettext" @default_domain "default" @po_wildcard "*/LC_MESSAGES/*.po" @doc false def __hash__(priv) do hash(po_files_in_priv(priv)) end defp hash(all_po_files) do all_po_files |> Enum.sort() |> :erlang.md5() end @doc false defmacro __before_compile__(env) do opts = Module.get_attribute(env.module, :gettext_opts) otp_app = Keyword.fetch!(opts, :otp_app) priv = Keyword.get(opts, :priv, @default_priv) all_po_files = po_files_in_priv(priv) hash_po_files = hash(all_po_files) known_po_files = known_po_files(all_po_files, opts) known_locales = Enum.map(known_po_files, & &1[:locale]) |> Enum.uniq() default_locale = opts[:default_locale] || quote(do: Application.fetch_env!(:gettext, :default_locale)) default_domain = opts[:default_domain] || @default_domain interpolation = opts[:interpolation] || Gettext.Interpolation.Default quote do @behaviour Gettext.Backend unquote(external_resources(known_po_files)) @doc false def __mix_recompile__? do unquote(hash_po_files) != Gettext.Compiler.__hash__(unquote(priv)) end # Info about the Gettext backend. @doc false def __gettext__(:priv), do: unquote(priv) def __gettext__(:otp_app), do: unquote(otp_app) def __gettext__(:known_locales), do: unquote(known_locales) def __gettext__(:default_locale), do: unquote(default_locale) def __gettext__(:default_domain), do: unquote(default_domain) def __gettext__(:interpolation), do: unquote(interpolation) if Gettext.Extractor.extracting?() do Gettext.ExtractorAgent.add_backend(__MODULE__) end # These are the two functions we generate inside the backend. @impl Gettext.Backend def lgettext(locale, domain, msgctxt \\ nil, msgid, bindings) @impl Gettext.Backend def lngettext(locale, domain, msgctxt \\ nil, msgid, msgid_plural, n, bindings) unquote(compile_po_files(env, known_po_files, opts)) # Catch-all clauses. def lgettext(locale, domain, msgctxt, msgid, bindings), do: handle_missing_translation(locale, domain, msgctxt, msgid, bindings) def lngettext(locale, domain, msgctxt, msgid, msgid_plural, n, bindings), do: handle_missing_plural_translation( locale, domain, msgctxt, msgid, msgid_plural, n, bindings ) end end defp external_resources(known_po_files) do Enum.map(known_po_files, fn po_file -> quote do @external_resource unquote(po_file.path) end end) end ## BEGIN of deprecated code ## TODO: Remove this block once "use Gettext" is removed defmacro generate_macros(_env) do quote unquote: false do defmacro dpgettext_noop(domain, msgctxt, msgid) do domain = Gettext.Compiler.expand_to_binary(domain, "domain", __MODULE__, __CALLER__) msgid = Gettext.Compiler.expand_to_binary(msgid, "msgid", __MODULE__, __CALLER__) msgctxt = Gettext.Compiler.expand_to_binary(msgctxt, "msgctxt", __MODULE__, __CALLER__) if Gettext.Extractor.extracting?() do Gettext.Extractor.extract( __CALLER__, __MODULE__, domain, msgctxt, msgid, Gettext.Compiler.get_and_flush_extracted_comments() ) end msgid end defmacro dgettext_noop(domain, msgid) do quote do unquote(__MODULE__).dpgettext_noop(unquote(domain), nil, unquote(msgid)) end end defmacro gettext_noop(msgid) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpgettext_noop(unquote(domain), nil, unquote(msgid)) end end defmacro pgettext_noop(msgid, context) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpgettext_noop(unquote(domain), unquote(context), unquote(msgid)) end end defmacro dpngettext_noop(domain, msgctxt, msgid, msgid_plural) do domain = Gettext.Compiler.expand_to_binary(domain, "domain", __MODULE__, __CALLER__) msgid = Gettext.Compiler.expand_to_binary(msgid, "msgid", __MODULE__, __CALLER__) msgctxt = Gettext.Compiler.expand_to_binary(msgctxt, "msgctxt", __MODULE__, __CALLER__) msgid_plural = Gettext.Compiler.expand_to_binary(msgid_plural, "msgid_plural", __MODULE__, __CALLER__) if Gettext.Extractor.extracting?() do Gettext.Extractor.extract( __CALLER__, __MODULE__, domain, msgctxt, {msgid, msgid_plural}, Gettext.Compiler.get_and_flush_extracted_comments() ) end {msgid, msgid_plural} end defmacro dngettext_noop(domain, msgid, msgid_plural) do quote do unquote(__MODULE__).dpngettext_noop( unquote(domain), nil, unquote(msgid), unquote(msgid_plural) ) end end defmacro pngettext_noop(msgctxt, msgid, msgid_plural) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpngettext_noop( unquote(domain), unquote(msgctxt), unquote(msgid), unquote(msgid_plural) ) end end defmacro ngettext_noop(msgid, msgid_plural) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpngettext_noop( unquote(domain), nil, unquote(msgid), unquote(msgid_plural) ) end end defmacro dpgettext(domain, msgctxt, msgid, bindings \\ Macro.escape(%{})) do quote do msgid = unquote(__MODULE__).dpgettext_noop(unquote(domain), unquote(msgctxt), unquote(msgid)) Gettext.dpgettext( unquote(__MODULE__), unquote(domain), unquote(msgctxt), msgid, unquote(bindings) ) end end defmacro dgettext(domain, msgid, bindings \\ Macro.escape(%{})) do quote do unquote(__MODULE__).dpgettext(unquote(domain), nil, unquote(msgid), unquote(bindings)) end end defmacro pgettext(msgctxt, msgid, bindings \\ Macro.escape(%{})) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpgettext( unquote(domain), unquote(msgctxt), unquote(msgid), unquote(bindings) ) end end defmacro gettext(msgid, bindings \\ Macro.escape(%{})) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpgettext(unquote(domain), nil, unquote(msgid), unquote(bindings)) end end defmacro dpngettext(domain, msgctxt, msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do quote do {msgid, msgid_plural} = unquote(__MODULE__).dpngettext_noop( unquote(domain), unquote(msgctxt), unquote(msgid), unquote(msgid_plural) ) Gettext.dpngettext( unquote(__MODULE__), unquote(domain), unquote(msgctxt), msgid, msgid_plural, unquote(n), unquote(bindings) ) end end defmacro dngettext(domain, msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do quote do unquote(__MODULE__).dpngettext( unquote(domain), nil, unquote(msgid), unquote(msgid_plural), unquote(n), unquote(bindings) ) end end defmacro ngettext(msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpngettext( unquote(domain), nil, unquote(msgid), unquote(msgid_plural), unquote(n), unquote(bindings) ) end end defmacro pngettext(msgctxt, msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do domain = __gettext__(:default_domain) quote do unquote(__MODULE__).dpngettext( unquote(domain), unquote(msgctxt), unquote(msgid), unquote(msgid_plural), unquote(n), unquote(bindings) ) end end defmacro gettext_comment(comment) do comment = Gettext.Compiler.expand_to_binary(comment, "comment", __MODULE__, __CALLER__) Gettext.Compiler.append_extracted_comment(comment) :ok end end end @doc false # TODO: remove me def expand_to_binary(term, what, gettext_module, env) when what in ~w(domain msgctxt msgid msgid_plural comment) do raiser = fn term -> raise ArgumentError, """ Gettext macros expect message keys (msgid and msgid_plural), domains, and comments to expand to strings at compile-time, but the given #{what} doesn't. This is what the macro received: #{inspect(term)} Dynamic messages should be avoided as they limit Gettext's ability to extract messages from your source code. If you are sure you need dynamic lookup, you can use the functions in the Gettext module: string = "hello world" Gettext.gettext(#{inspect(gettext_module)}, string) """ end # We support nil too in order to fall back to a nil context and always use the *p # variants of the Gettext macros. case Macro.expand(term, env) do term when is_binary(term) or is_nil(term) -> term {:<<>>, _, pieces} = term -> if Enum.all?(pieces, &is_binary/1), do: Enum.join(pieces), else: raiser.(term) other -> raiser.(other) end end @doc false def append_extracted_comment(comment) do existing = Process.get(:gettext_comments, []) Process.put(:gettext_comments, [" " <> comment | existing]) :ok end @doc false def get_and_flush_extracted_comments() do Enum.reverse(Process.delete(:gettext_comments) || []) end ## END of deprecated block @doc """ Logs a warning via `Logger.error/1` if `domain` contains slashes. This function is called by `lgettext` and `lngettext`. It could make sense to make this function raise an error since slashes in domains are not supported, but we decided not to do so and to only emit a warning since the expected behaviour for Gettext functions/macros when the domain or message is not known is to return the original string (msgid) and raising here would break that contract. """ @spec warn_if_domain_contains_slashes(binary) :: :ok def warn_if_domain_contains_slashes(domain) do if String.contains?(domain, "/") do _ = Logger.error(fn -> ["Slashes in domains are not supported: ", inspect(domain)] end) end :ok end # Compiles all the `.po` files in the given directory (`dir`) into `lgettext/4` # and `lngettext/6` function clauses. defp compile_po_files(env, known_po_files, opts) do plural_mod = Keyword.get(opts, :plural_forms) || Application.get_env(:gettext, :plural_forms, Gettext.Plural) opts = if opts[:one_module_per_locale] do IO.warn( ":one_module_per_locale is deprecated, please use split_module_by: [:locale] instead" ) Keyword.put_new(opts, :split_module_by, [:locale]) else opts end case List.wrap(opts[:split_module_by]) do [] -> Enum.map( known_po_files, &compile_unified_po_file(env, &1, plural_mod, opts[:interpolation]) ) split -> grouped = Enum.group_by(known_po_files, &split_module_name(env, &1, split)) case Keyword.get(opts, :split_module_compilation, :parallel) do :serial -> Enum.map(grouped, fn {module, files} -> compile_split_po_files(env, module, files, plural_mod, opts[:interpolation]) end) :parallel -> grouped |> Enum.map(fn {module, files} -> Kernel.ParallelCompiler.async(fn -> compile_split_po_files(env, module, files, plural_mod, opts[:interpolation]) end) end) |> Enum.map(fn task -> Task.await(task, :infinity) end) end end end defp split_module_name(env, po_file, split) do String.to_atom( "#{env.module}.T" <> if(:locale in split, do: "_" <> po_file.locale, else: "") <> if(:domain in split, do: "_" <> po_file.domain, else: "") ) end defp compile_unified_po_file(env, po_file, plural_mod, interpolation_module) do {locale, domain, singular_fun, plural_fun, quoted} = compile_po_file(:defp, po_file, env, plural_mod, interpolation_module) quote do unquote(quoted) def lgettext(unquote(locale), unquote(domain), msgctxt, msgid, bindings) do unquote(singular_fun)(msgctxt, msgid, bindings) end def lngettext(unquote(locale), unquote(domain), msgctxt, msgid, msgid_plural, n, bindings) do unquote(plural_fun)(msgctxt, msgid, msgid_plural, n, bindings) end end end defp compile_split_po_files(env, module, files, plural_mod, interpolation_module) do {current, split} = Enum.reduce( files, {[], []}, &compile_split_po_file(env, module, plural_mod, &1, interpolation_module, &2) ) create_split_module(env, module, split) current end defp compile_split_po_file(env, module, plural_mod, po_file, interpolation_module, {acc1, acc2}) do {locale, domain, singular_fun, plural_fun, split_module_quoted} = compile_po_file(:def, po_file, env, plural_mod, interpolation_module) current_module_quoted = quote do def lgettext(unquote(locale), unquote(domain), msgctxt, msgid, bindings) do unquote(module).unquote(singular_fun)(msgctxt, msgid, bindings) end def lngettext(unquote(locale), unquote(domain), msgctxt, msgid, msgid_plural, n, bindings) do unquote(module).unquote(plural_fun)(msgctxt, msgid, msgid_plural, n, bindings) end end {[current_module_quoted | acc1], [split_module_quoted | acc2]} end defp create_split_module(env, module, messages) do exprs = [quote(do: @moduledoc(false)) | messages] Module.create(module, block(exprs), env) :ok end # Compiles a .po file into a list of lgettext/5 (for messages) and # lngettext/7 (for plural messages) clauses. defp compile_po_file(kind, po_file, env, plural_mod, interpolation_module) do %{locale: locale, domain: domain, path: path} = po_file %Messages{messages: messages, file: file} = messages_struct = PO.parse_file!(path, strip_meta: true) plural_forms_fun = :"#{locale}_#{domain}_plural" plural_forms = compile_plural_forms(locale, messages_struct, plural_mod, plural_forms_fun) nplurals = nplurals(locale, messages_struct, plural_mod) singular_fun = :"#{locale}_#{domain}_lgettext" plural_fun = :"#{locale}_#{domain}_lngettext" messages = Enum.filter(messages, &match?(%{obsolete: false}, &1)) mapper = &compile_message( kind, locale, &1, singular_fun, plural_fun, file, {plural_forms_fun, nplurals}, interpolation_module ) messages = block(Enum.map(messages, mapper)) quoted = quote do unquote(plural_forms) unquote(messages) Kernel.unquote(kind)(unquote(singular_fun)(msgctxt, msgid, bindings)) do unquote(env.module).handle_missing_translation( unquote(locale), unquote(domain), msgctxt, msgid, bindings ) end Kernel.unquote(kind)(unquote(plural_fun)(msgctxt, msgid, msgid_plural, n, bindings)) do unquote(env.module).handle_missing_plural_translation( unquote(locale), unquote(domain), msgctxt, msgid, msgid_plural, n, bindings ) end end {locale, domain, singular_fun, plural_fun, quoted} end defp nplurals(locale, messages_struct, plural_mod) do plural_mod.nplurals(Plural.plural_info(locale, messages_struct, plural_mod)) end defp compile_plural_forms(locale, messages_struct, plural_mod, plural_fun) do quote do defp unquote(plural_fun)(n) do unquote(plural_mod).plural( unquote(Macro.escape(Plural.plural_info(locale, messages_struct, plural_mod))), n ) end end end defp locale_and_domain_from_path(path) do [file, "LC_MESSAGES", locale | _rest] = path |> Path.split() |> Enum.reverse() domain = Path.rootname(file, ".po") {locale, domain} end defp compile_message( kind, _locale, %Message.Singular{} = message, singular_fun, _plural_fun, _file, _pluralization, interpolation_module ) do msgid = IO.iodata_to_binary(message.msgid) msgstr = IO.iodata_to_binary(message.msgstr) msgctxt = message.msgctxt && IO.iodata_to_binary(message.msgctxt) case msgstr do # Only actually generate this function clause if the msgstr is not empty. # If it is empty, it will trigger the missing message case. "" -> nil _ -> quote do Kernel.unquote(kind)( unquote(singular_fun)(unquote(msgctxt), unquote(msgid), bindings) ) do require unquote(interpolation_module) unquote(interpolation_module).compile_interpolate( :translation, unquote(msgstr), bindings ) end end end end defp compile_message( kind, locale, %Message.Plural{} = message, singular_fun, plural_fun, file, {plural_forms_fun, nplurals}, interpolation_module ) do warn_if_missing_plural_forms(locale, nplurals, message, file) msgid = IO.iodata_to_binary(message.msgid) msgid_plural = IO.iodata_to_binary(message.msgid_plural) msgstr = Enum.map(message.msgstr, fn {form, str} -> {form, IO.iodata_to_binary(str)} end) msgctxt = message.msgctxt && IO.iodata_to_binary(message.msgctxt) # If any of the msgstrs is empty, then we skip the generation of this # function clause. The reason we do this is the same as for the # `%Message.Singular{}` clause. unless Enum.any?(msgstr, &match?({_form, ""}, &1)) do # We use flat_map here because clauses can only be defined in blocks, # so when quoted they are a list. clauses = Enum.flat_map(msgstr, fn {form, str} -> quote do unquote(form) -> require unquote(interpolation_module) unquote(interpolation_module).compile_interpolate( :plural_translation, unquote(str), var!(bindings) ) end end) error_clause = quote do form -> raise Gettext.PluralFormError, form: form, locale: unquote(locale), file: unquote(file), line: unquote(Message.source_line_number(message, :msgid)) end singular_fun_impl = msgstr |> Enum.find(&match?({0, _msgstr}, &1)) |> case do {0, ""} -> nil {0, msgstr} -> quote do Kernel.unquote(kind)( unquote(singular_fun)(unquote(msgctxt), unquote(msgid), bindings) ) do require unquote(interpolation_module) unquote(interpolation_module).compile_interpolate( :translation, unquote(msgstr), bindings ) end end nil -> nil end plural_fun_impl = quote generated: true do Kernel.unquote(kind)( unquote(plural_fun)( unquote(msgctxt), unquote(msgid), unquote(msgid_plural), n, bindings ) ) do plural_form = unquote(plural_forms_fun)(n) var!(bindings) = Map.put(bindings, :count, n) case plural_form, do: unquote(clauses ++ error_clause) end end quote do unquote(singular_fun_impl) unquote(plural_fun_impl) end end end defp warn_if_missing_plural_forms(locale, nplurals, message, file) do Enum.each(0..(nplurals - 1), fn form -> unless Map.has_key?(message.msgstr, form) do _ = Logger.error([ "#{file}:#{Message.source_line_number(message, :msgid)}: message is missing plural form ", Integer.to_string(form), " which is required by the locale ", inspect(locale) ]) end end) end defp block(contents) when is_list(contents) do {:__block__, [], contents} end defp po_files_in_priv(priv) do priv |> Path.join(@po_wildcard) |> Path.wildcard() end # Returns the known the PO files in `messages_dir` with their locale and domain # If allowed_locales is configured, it removes all the PO files that do not belong # to those locales defp known_po_files(all_po_files, opts) do all_po_files |> Enum.map(fn path -> {locale, domain} = locale_and_domain_from_path(path) %{locale: locale, path: path, domain: domain} end) |> maybe_restrict_locales(opts[:allowed_locales]) end defp maybe_restrict_locales(po_files, nil) do po_files end defp maybe_restrict_locales(po_files, allowed_locales) when is_list(allowed_locales) do allowed_locales = MapSet.new(Enum.map(allowed_locales, &to_string/1)) Enum.filter(po_files, &MapSet.member?(allowed_locales, &1[:locale])) end end ================================================ FILE: lib/gettext/error.ex ================================================ defmodule Gettext.Error do @moduledoc """ A generic error raised for a variety of possible Gettext-related reasons. """ @typedoc since: "0.22.0" @type t() :: %__MODULE__{} defexception [:message] end ================================================ FILE: lib/gettext/extractor.ex ================================================ defmodule Gettext.Extractor do @moduledoc false # This module is responsible for extracting messages (it's called from the # *gettext macros) and dumping those messages to POT files, merging with # existing POT files if necessary. # # ## Ordering # # Ordering is mostly taken care of in merge_template/2, where we go over the # messages in an existing POT file and merge them if necessary (thus # keeping the order from the original file), then adding the messages from # the new in-memory POT (sorted by name). alias Gettext.Error alias Gettext.ExtractorAgent alias Gettext.Merger alias Expo.PO alias Expo.Message alias Expo.Messages @extracted_messages_flag "elixir-autogen" @new_pot_comment String.split( """ # This file is a PO Template file. # # "msgid"s here are often extracted from source code. # Add new messages manually only if they're dynamic # messages that can't be statically extracted. # # Run "mix gettext.extract" to bring this file up to # date. Leave "msgstr"s empty as changing them here has no # effect: edit them in PO (.po) files instead. """, "\n" ) @doc """ Enables message extraction. """ @spec enable() :: :ok def enable() do ExtractorAgent.enable() end @doc """ Disables extraction. """ @spec disable() :: :ok def disable() do ExtractorAgent.disable() end @doc """ Tells whether messages are being extracted. """ @spec extracting?() :: boolean def extracting?() do # Because the extractor agent may not be enabled during compilation # time (as it requires the optional Gettext compiler), we need to # check if the agent is up and running before querying it. Process.whereis(ExtractorAgent) && ExtractorAgent.extracting?() end @doc """ Extracts a message by temporarily storing it in an agent. Note that this function doesn't perform any operation on the filesystem. """ @spec extract( Macro.Env.t(), backend :: module, domain :: binary | :default, msgctxt :: binary, id :: binary | {binary, binary}, extracted_comments :: [binary] ) :: :ok def extract(%Macro.Env{} = caller, backend, domain, msgctxt, id, extracted_comments) do format_flag = backend.__gettext__(:interpolation).message_format() domain = case domain do :default -> backend.__gettext__(:default_domain) string when is_binary(string) -> string end message = create_message_struct( id, msgctxt, caller.file, caller.line, extracted_comments, format_flag ) ExtractorAgent.add_message(backend, domain, message) end @doc """ Returns a list of POT files based on the results of the extraction. Returns a list of paths and their contents to be written to disk. Existing POT files are either purged from obsolete messages (in case no extracted message ends up in that file) or merged with the extracted messages; new POT files are returned for extracted messages that belong to a POT file that doesn't exist yet. This is a stateful operation. Once pot_files are generated, their information is permanently removed from the extractor. """ @spec pot_files(atom, Keyword.t()) :: [{path :: String.t(), contents :: iodata}] def pot_files(app, gettext_config) do backends = ExtractorAgent.pop_backends(app) warn_on_conflicting_backends(backends) existing_pot_files = pot_files_for_backends(backends) backends |> ExtractorAgent.pop_message() |> create_po_structs_from_extracted_messages() |> merge_pot_files(existing_pot_files, gettext_config) end defp warn_on_conflicting_backends(backends) do Enum.reduce(backends, %{}, fn backend, acc -> priv = backend.__gettext__(:priv) case acc do %{^priv => other_backend} -> IO.warn( "the Gettext backend #{inspect(backend)} has the same :priv directory as " <> "#{inspect(other_backend)}, which means they will override each other. " <> "Please set the :priv option to different directories or use Gettext " <> "inside each backend" ) acc %{} -> Map.put(acc, priv, backend) end end) end # Returns all the .pot files for each of the given `backends`. defp pot_files_for_backends(backends) do Enum.flat_map(backends, fn backend -> backend.__gettext__(:priv) |> Path.join("**/*.pot") |> Path.wildcard() end) end # This returns a list of {absolute_path, %Gettext.PO{}} tuples. # `all_messages` looks like this: # # %{MyBackend => %{"a_domain" => %{"a message id" => a_message}}} # defp create_po_structs_from_extracted_messages(all_messages) do for {backend, domains} <- all_messages, {domain, messages} <- domains do messages = Map.values(messages) {pot_path(backend, domain), po_struct_from_messages(messages)} end end defp pot_path(backend, domain) do Path.join(backend.__gettext__(:priv), "#{domain}.pot") end defp po_struct_from_messages(messages) do # Sort all the messages and the references of each message in order # to make as few changes as possible to the PO(T) files. messages = messages |> Enum.sort_by(&Message.key/1) |> Enum.map(&sort_references/1) %Messages{messages: messages, top_comments: @new_pot_comment, headers: [""]} end defp sort_references(message) do update_in(message.references, &Enum.sort/1) end defp create_message_struct( {msgid, msgid_plural}, msgctxt, file, line, extracted_comments, format_flag ) do %Message.Plural{ msgid: [msgid], msgctxt: if(msgctxt != nil, do: [msgctxt], else: nil), msgid_plural: [msgid_plural], msgstr: %{0 => [""], 1 => [""]}, flags: [[@extracted_messages_flag, format_flag]], references: [[{Path.relative_to_cwd(file), line}]], extracted_comments: extracted_comments } end defp create_message_struct(msgid, msgctxt, file, line, extracted_comments, format_flag) do %Message.Singular{ msgid: [msgid], msgctxt: if(msgctxt != nil, do: [msgctxt], else: nil), msgstr: [""], flags: [[@extracted_messages_flag, format_flag]], references: [[{Path.relative_to_cwd(file), line}]], extracted_comments: extracted_comments } end # Made public for testing. @doc false def merge_pot_files(po_structs, pot_files, gettext_config) do # pot_files is a list of paths to existing .pot files while po_structs is a # list of {path, struct} for new %Gettext.PO{} structs that we have # extracted. If we turn pot_files into a list of {path, whatever} tuples, # then we can take advantage of Map.merge/3 to find files that we have to # update, delete, or add. pot_files = Map.new(pot_files, &{&1, :existing}) po_structs = Map.new(po_structs, fn {path, struct} -> {path, Merger.prune_references(struct, gettext_config)} end) # After Map.merge/3, we have something like: # %{path => {:merged, :unchanged | %Messages{}}, path => %Messages{}, path => :existing} # and after mapping tag_files/1 over that we have something like: # %{path => {:merged, :unchanged | %Messages{}}, path => {:unmerged, :unchanged | %Messages{}}, path => {:new, %Messages{}}} Map.merge(pot_files, po_structs, &merge_existing_and_extracted(&1, &2, &3, gettext_config)) |> Enum.map(&tag_files(&1, gettext_config)) |> Enum.map(&dump_tagged_file/1) end # This function is called by merge_pot_files/2 as the function passed to # Map.merge/3 (so when we have both an :existing file and a new extracted # in-memory PO struct both located at "path"). defp merge_existing_and_extracted(path, :existing, extracted, gettext_config) do {:merged, merge_or_unchanged(path, extracted, gettext_config)} end # Returns :unchanged if merging `existing_path` with `new_po` changes nothing, # otherwise a %Gettext.PO{} struct with the changed contents. defp merge_or_unchanged(existing_path, new_po, gettext_config) do {existing_contents, existing_po} = read_contents_and_parse(existing_path) merged_po = merge_template(existing_po, new_po, gettext_config) if IO.iodata_to_binary(PO.compose(merged_po)) == existing_contents do :unchanged else merged_po end end defp read_contents_and_parse(path) do contents = File.read!(path) {contents, PO.parse_file!(path, file: path)} end # This function "tags" a {path, _} tuple in order to distinguish POT files # that have been merged (one existed at `path` and there's a new one to put at # `path` as well), POT files that exist but have no new counterpart (`{path, # :existing}`) and new files that do not exist yet. # These are marked as: # * {path, {:merged, _}} - one existed and there's a new one # * {path, {:unmerged, _}} - one existed, no new one # * {path, {:new, _}} - none existed, there's a new one # Note that existing files with no new corresponding file are "pruned", for example, # merged with an empty %Messages{} struct to remove obsolete message (see # prune_unmerged/1), because the user could still have PO message that # they manually inserted in that file. defp tag_files({_path, {:merged, _}} = entry, _gettext_config), do: entry defp tag_files({path, :existing}, gettext_config), do: {path, {:unmerged, prune_unmerged(path, gettext_config)}} defp tag_files({path, new_po}, _gettext_config), do: {path, {:new, new_po}} # This function "dumps" merged files and unmerged files without any changes, # and dumps new POT files adding an informative comment to them. This doesn't # write anything to disk, it just returns `{path, contents}` tuples. defp dump_tagged_file({path, {_tag, :unchanged}}), do: {path, :unchanged} defp dump_tagged_file({path, {_tag, po}}), do: {path, {:changed, PO.compose(po)}} defp prune_unmerged(path, gettext_config) do merge_or_unchanged(path, %Messages{messages: []}, gettext_config) end # Merges a %Messages{} struct representing an existing POT file with an # in-memory-only %Messages{} struct representing the new POT file. # Made public for testing. @doc false def merge_template(existing, new, gettext_config) do protected_pattern = gettext_config[:excluded_refs_from_purging] # We go over the existing message in order so as to keep the existing # order as much as possible. old_and_merged = Enum.flat_map(existing.messages, fn message -> cond do same = Messages.find(new, message) -> [merge_message(message, same)] protected?(message, protected_pattern) -> [message] autogenerated?(message) -> [] true -> [message] end end) # We reject all messages that appear in `existing` so that we're left # with the messages that only appear in `new`. unique_new = Enum.reject(new.messages, &Messages.find(existing, &1)) messages = old_and_merged ++ unique_new sort_by_msgid = case gettext_config[:sort_by_msgid] || false do val when val in [:case_sensitive, :case_insensitive, false] -> val true -> IO.warn(""" Passing "true" to the :sort_by_msgid option is deprecated. \ Use :case_sensitive instead, or specify :case_insensitive.\ """) :case_sensitive end messages = case sort_by_msgid do :case_sensitive -> Enum.sort_by(messages, &IO.chardata_to_string(&1.msgid)) :case_insensitive -> Enum.sort_by(messages, &String.downcase(IO.chardata_to_string(&1.msgid))) false -> messages end %Messages{ messages: messages, headers: existing.headers, top_comments: existing.top_comments } end defp merge_message(old, new) do ensure_empty_msgstr!(old) ensure_empty_msgstr!(new) # Take all flags from `old` and only the `@extracted_messages_flag` flag from `new` # to avoid re-adding manually removed flags. flags = if Message.has_flag?(new, @extracted_messages_flag) do Message.append_flag(old, @extracted_messages_flag).flags else old.flags end old |> Message.merge(new) |> Map.merge(%{ flags: flags, # We don't care about the references of the old message since the new # in-memory message has all the actual and current references. references: new.references, extracted_comments: new.extracted_comments }) end defp ensure_empty_msgstr!(%Message.Singular{msgstr: msgstr} = message) do unless blank?(msgstr) do raise Error, "message with msgid '#{IO.iodata_to_binary(message.msgid)}' has a non-empty msgstr" end end defp ensure_empty_msgstr!(%Message.Plural{msgstr: msgstr} = message) do if Enum.any?(Map.values(msgstr), &(not blank?(&1))) do raise Error, "plural message with msgid '#{IO.iodata_to_binary(message.msgid)}' has a non-empty msgstr" end end defp ensure_empty_msgstr!(%Message.Plural{} = message) do raise Error, "plural message with msgid '#{IO.iodata_to_binary(message.msgid)}' has a non-empty msgstr" end defp blank?(str) when not is_nil(str), do: IO.iodata_length(str) == 0 defp blank?(_), do: true @spec autogenerated?(message :: Message.t()) :: boolean defp autogenerated?(message) do Message.has_flag?(message, "elixir-autogen") end # A message that is protected from purging will never be removed by Gettext. # Which messages are proteced can be configured using Mix. @spec protected?(message :: Message.t(), protected_pattern :: Regex.t()) :: boolean defp protected?(_t, nil), do: false defp protected?(%{references: []}, _pattern), do: false defp protected?(%{references: refs}, pattern), do: refs |> List.flatten() |> Enum.any?(fn {path, _} -> Regex.match?(pattern, path) end) end ================================================ FILE: lib/gettext/extractor_agent.ex ================================================ defmodule Gettext.ExtractorAgent do @moduledoc false use Agent require Logger alias Expo.Message @name __MODULE__ # :messages is a map where keys are Gettext backends and values # are maps. In these maps, keys are domains and values are maps of # message_id => message. # :backends is just a list of backends that call `use Gettext`. @initial_state %{ messages: %{}, backends: [], extracting?: false } @spec start_link(keyword()) :: Agent.on_start() def start_link([] = _opts) do Agent.start_link(fn -> @initial_state end, name: @name) end @spec enable() :: :ok def enable() do Agent.update(@name, &put_in(&1.extracting?, true)) end @spec disable() :: :ok def disable() do Agent.update(@name, &put_in(&1.extracting?, false)) end @spec extracting?() :: boolean() def extracting?() do Agent.get(@name, & &1.extracting?) end @spec add_message(backend :: module(), domain :: String.t(), Message.t()) :: :ok def add_message(backend, domain, message) do key = Message.key(message) Agent.cast(@name, fn state -> # Initialize the given backend to an empty map if it wasn't there. state = update_in(state.messages, &Map.put_new(&1, backend, %{})) update_in(state.messages[backend][domain], fn messages -> Map.update(messages || %{}, key, message, &merge_messages(&1, message)) end) end) end def add_backend(backend) do Agent.cast(@name, fn state -> update_in(state.backends, &[backend | &1]) end) end def stop() do Agent.stop(@name) end def pop_message(backends) do Agent.get_and_update(@name, fn state -> get_and_update_in(state.messages, &Map.split(&1, backends)) end) end def pop_backends(app) do Agent.get_and_update(@name, fn state -> get_and_update_in(state.backends, fn backends -> Enum.split_with(backends, &(&1.__gettext__(:otp_app) == app)) end) end) end defp merge_messages(%Message.Singular{} = message_1, %Message.Plural{} = message_2) do # Flipping the arguments to make sure that the pluaral message (more information) is used as the base message merge_messages(message_2, message_1) end defp merge_messages(%Message.Plural{} = message_1, %Message.Plural{} = message_2) do # Make sure message choice is deterministic [message_1, message_2] = Enum.sort_by([message_1, message_2], &IO.iodata_to_binary(&1.msgid_plural)) if IO.iodata_to_binary(message_1.msgid_plural) != IO.iodata_to_binary(message_2.msgid_plural) do Logger.warning(""" Plural message for '#{IO.iodata_to_binary(message_1.msgid)}' is not matching: Using '#{IO.iodata_to_binary(message_2.msgid_plural)}' instead of '#{IO.iodata_to_binary(message_1.msgid_plural)}'. References: #{dump_references(message_1.references ++ message_2.references)}\ """) end merge_messages_after_checks(message_1, message_2) end defp merge_messages(message_1, message_2), do: merge_messages_after_checks(message_1, message_2) defp merge_messages_after_checks(message_1, message_2) do message_1 |> Map.put(:references, message_1.references ++ message_2.references) |> Map.put( :extracted_comments, Enum.uniq(message_1.extracted_comments ++ message_2.extracted_comments) ) end defp dump_references(references) do references |> List.flatten() |> Enum.map(fn {file, line} -> [file, ":", Integer.to_string(line)] file -> file end) |> Enum.intersperse(", ") end end ================================================ FILE: lib/gettext/fuzzy.ex ================================================ defmodule Gettext.Fuzzy do @moduledoc false alias Expo.Message @type message_key :: {binary | nil, binary | {binary, binary}} @doc """ Returns a matcher function that takes two message keys and checks if they match. `String.jaro_distance/2` (which calculates the Jaro distance) is used to measure the distance between the two messages. `threshold` is the minimum distance that means a match. `{:match, distance}` is returned in case of a match, `:nomatch` otherwise. """ @spec matcher(float) :: (message_key, message_key -> {:match, float} | :nomatch) def matcher(threshold) do fn old_key, new_key -> distance = jaro_distance(old_key, new_key) if distance >= threshold, do: {:match, distance}, else: :nomatch end end @doc """ Finds the Jaro distance between the msgids of two messages. To mimic the behaviour of the `msgmerge` tool, this function only calculates the Jaro distance of the msgids of the two messages, even if one (or both) of them is a plural message. As per `msgmerge`, the msgctxt of a message is completely ignored when calculating the distance. """ @spec jaro_distance(message_key, message_key) :: float def jaro_distance({_context1, key1}, {_context2, key2}) do jaro_distance_on_key(key1, key2) end # Apparently, msgmerge only looks at the msgid when performing fuzzy # matching. This means that if we have two plural messages with similar # msgids but very different msgid_plurals, they'll still fuzzy match. def jaro_distance_on_key(key1, key2) when is_binary(key1) and is_binary(key2), do: String.jaro_distance(key1, key2) def jaro_distance_on_key({key1, _}, key2) when is_binary(key2), do: String.jaro_distance(key1, key2) def jaro_distance_on_key(key1, {key2, _}) when is_binary(key1), do: String.jaro_distance(key1, key2) def jaro_distance_on_key({key1, _}, {key2, _}), do: String.jaro_distance(key1, key2) @doc """ Merges a message with the corresponding fuzzy match. `new` is the newest message and `existing` is the existing message that we use to populate the msgstr of the newest message. Note that if `new` is a regular message, then the result will be a regular message; if `new` is a plural message, then the result will be a plural message. """ @spec merge(new :: Message.t(), existing :: Message.t()) :: Message.t() def merge(new, existing) do # Everything comes from "new", except for the msgstr and the comments. new |> Map.put(:comments, existing.comments) |> merge_msgstr(existing) |> Message.append_flag("fuzzy") end defp merge_msgstr(%Message.Singular{} = new, %Message.Singular{} = existing), do: %{new | msgstr: existing.msgstr} defp merge_msgstr(%Message.Singular{} = new, %Message.Plural{} = existing), do: %{new | msgstr: existing.msgstr[0]} defp merge_msgstr(%Message.Plural{} = new, %Message.Singular{} = existing), do: %{new | msgstr: Map.new(new.msgstr, fn {i, _} -> {i, existing.msgstr} end)} defp merge_msgstr(%Message.Plural{} = new, %Message.Plural{} = existing), do: %{new | msgstr: existing.msgstr} end ================================================ FILE: lib/gettext/interpolation/default.ex ================================================ defmodule Gettext.Interpolation.Default do @moduledoc """ Default implementation for the `Gettext.Interpolation` behaviour. Replaces `%{binding_name}` with the string value of the `binding_name` binding. """ @behaviour Gettext.Interpolation @typedoc """ Something that can be interpolated. It's either a string (a literal) or an atom (representing a binding name). """ @type interpolatable() :: [String.t() | atom()] # Extracts interpolations from a given string. # This function extracts all interpolations in the form `%{interpolation}` # contained inside `str`, converts them to atoms and then returns a list of # string and interpolation keys. @doc false @spec to_interpolatable(String.t()) :: interpolatable() def to_interpolatable(string) when is_binary(string) do start_pattern = :binary.compile_pattern("%{") end_pattern = :binary.compile_pattern("}") string |> to_interpolatable(_current = "", _acc = [], start_pattern, end_pattern) |> Enum.reverse() end defp to_interpolatable(string, current, acc, start_pattern, end_pattern) do case :binary.split(string, start_pattern) do # If we have one element, no %{ was found so this is the final part of the # string. [rest] -> prepend_if_not_empty(current <> rest, acc) # If we found a %{ but it's followed by an immediate }, then we just # append %{} to the current string and keep going. [before, "}" <> rest] -> new_current = current <> before <> "%{}" to_interpolatable(rest, new_current, acc, start_pattern, end_pattern) # Otherwise, we found the start of a binding. [before, binding_and_rest] -> case :binary.split(binding_and_rest, end_pattern) do # If we don't find the end of this binding, it means we're at a string # like "foo %{ no end". In this case we consider no bindings to be # there. [_] -> [current <> string | acc] # This is the case where we found a binding, so we put it in the acc # and keep going. [binding, rest] -> new_acc = [String.to_atom(binding) | prepend_if_not_empty(before, acc)] to_interpolatable(rest, "", new_acc, start_pattern, end_pattern) end end end defp prepend_if_not_empty("", list), do: list defp prepend_if_not_empty(string, list), do: [string | list] @doc """ Interpolate a message or interpolatable with the given bindings. Implementation of the `c:Gettext.Interpolation.runtime_interpolate/2` callback. This function takes a message and some bindings and returns an `{:ok, interpolated_string}` tuple if interpolation is successful. If it encounters a binding in the message that is missing from `bindings`, it returns `{:missing_bindings, incomplete_string, missing_bindings}` where `incomplete_string` is the string with only the present bindings interpolated and `missing_bindings` is a list of atoms representing bindings that are in `interpolatable` but not in `bindings`. ## Examples iex> msgid = "Hello %{name}, you have %{count} unread messages" iex> good_bindings = %{name: "José", count: 3} iex> Gettext.Interpolation.Default.runtime_interpolate(msgid, good_bindings) {:ok, "Hello José, you have 3 unread messages"} iex> Gettext.Interpolation.Default.runtime_interpolate(msgid, %{name: "José"}) {:missing_bindings, "Hello José, you have %{count} unread messages", [:count]} iex> msgid = "Hello %{name}, you have %{count} unread messages" iex> interpolatable = Gettext.Interpolation.Default.to_interpolatable(msgid) iex> good_bindings = %{name: "José", count: 3} iex> Gettext.Interpolation.Default.runtime_interpolate(interpolatable, good_bindings) {:ok, "Hello José, you have 3 unread messages"} iex> Gettext.Interpolation.Default.runtime_interpolate(interpolatable, %{name: "José"}) {:missing_bindings, "Hello José, you have %{count} unread messages", [:count]} """ @impl true def runtime_interpolate(message, bindings) def runtime_interpolate(message, %{} = bindings) when is_binary(message) do message |> to_interpolatable() |> runtime_interpolate(bindings) end def runtime_interpolate(interpolatable, %{} = bindings) when is_list(interpolatable) do interpolate(interpolatable, bindings, [], []) end defp interpolate([string | segments], bindings, strings, missing) when is_binary(string) do interpolate(segments, bindings, [string | strings], missing) end defp interpolate([atom | segments], bindings, strings, missing) when is_atom(atom) do case bindings do %{^atom => value} -> interpolate(segments, bindings, [to_string(value) | strings], missing) %{} -> strings = ["%{" <> Atom.to_string(atom) <> "}" | strings] interpolate(segments, bindings, strings, [atom | missing]) end end defp interpolate([], _bindings, strings, []) do {:ok, IO.iodata_to_binary(Enum.reverse(strings))} end defp interpolate([], _bindings, strings, missing) do missing = missing |> Enum.reverse() |> Enum.uniq() {:missing_bindings, IO.iodata_to_binary(Enum.reverse(strings)), missing} end # Returns all the interpolation keys contained in the given string or list of # segments. # This function returns a list of all the interpolation keys (patterns in the # form `%{interpolation}`) contained in its argument. # If the argument is a segment list, that is, a list of strings and atoms where # atoms represent interpolation keys, then only the atoms in the list are # returned. @doc false @spec keys(String.t() | interpolatable()) :: [atom()] def keys(string_or_interpolatable) def keys(string) when is_binary(string), do: string |> to_interpolatable() |> keys() def keys(interpolatable) when is_list(interpolatable), do: interpolatable |> Enum.filter(&is_atom/1) |> Enum.uniq() @doc """ Compiles a static message to interpolate with dynamic bindings. Implementation of the `c:Gettext.Interpolation.compile_interpolate/3` macro callback. Takes a static message and some dynamic bindings. The generated code will return an `{:ok, interpolated_string}` tuple if the interpolation is successful. If it encounters a binding in the message that is missing from `bindings`, it returns `{:missing_bindings, incomplete_string, missing_bindings}`, where `incomplete_string` is the string with only the present bindings interpolated and `missing_bindings` is a list of atoms representing bindings that are in `interpolatable` but not in `bindings`. """ @impl true defmacro compile_interpolate(message_type, message, bindings) do unless is_binary(message) do raise """ #{inspect(__MODULE__)}.compile_interpolate/2 can only be used at compile time with \ static messages. Alternatively, use #{inspect(__MODULE__)}.runtime_interpolate/2. """ end interpolatable = to_interpolatable(message) keys = keys(interpolatable) match_clause = match_clause(keys) compile_string = compile_string(interpolatable) case {keys, message_type} do # If no keys are in the message, the message can be returned without interpolation {[], _message_type} -> quote do: {:ok, unquote(message)} # If the message only contains the key `count` and it is a plural message, # gettext ensures that `count` is always set. Therefore the dynamic interpolation # will never be needed. {[:count], :plural_translation} -> quote do unquote(match_clause) = unquote(bindings) {:ok, unquote(compile_string)} end {_keys, _message_type} -> quote generated: true do case unquote(bindings) do unquote(match_clause) -> {:ok, unquote(compile_string)} %{} = other_bindings -> unquote(__MODULE__).runtime_interpolate(unquote(interpolatable), other_bindings) end end end end # Compiles a list of atoms into a "match" map. For example `[:foo, :bar]` gets # compiled to `%{foo: foo, bar: bar}`. All generated variables are under the # current `__MODULE__`. defp match_clause(keys) do {:%{}, [], Enum.map(keys, &{&1, Macro.var(&1, __MODULE__)})} end # Compiles a string into a binary with `%{var}` patterns turned into `var` # variables, namespaced inside the current `__MODULE__`. defp compile_string(interpolatable) do parts = Enum.map(interpolatable, fn key when is_atom(key) -> quote do: to_string(unquote(Macro.var(key, __MODULE__))) :: binary str -> str end) {:<<>>, [], parts} end @doc """ Implementation of `c:Gettext.Interpolation.message_format/0`. ## Examples iex> Gettext.Interpolation.Default.message_format() "elixir-format" """ @impl true def message_format, do: "elixir-format" end ================================================ FILE: lib/gettext/interpolation.ex ================================================ defmodule Gettext.Interpolation do @moduledoc """ Behaviour to provide Gettext string interpolation. By default, Gettext uses `Gettext.Interpolation.Default` as the interpolation module. """ @typedoc since: "0.19.0" @type translation_type() :: :translation | :plural_translation @typedoc since: "0.22.0" @type bindings() :: %{optional(atom()) => term()} @doc """ Called to perform interpolation *at runtime*. If successful, should return `{:ok, interpolated_string}`. If there are missing bindings, should return `{:missing_bindings, partially_interpolated, missing}` where `partially_interpolated` is a string with the available bindings interpolated. """ @doc since: "0.19.0" @callback runtime_interpolate(message :: String.t(), bindings()) :: {:ok, String.t()} | {:missing_bindings, partially_interpolated_message :: String.t(), missing_bindings :: [atom()]} @doc """ Called to perform interpolation *at compile time*. """ @doc since: "0.19.0" @macrocallback compile_interpolate(translation_type(), message :: String.t(), bindings()) :: Macro.t() @doc """ Defines the Gettext message format to be used when extracting. The default interpolation module that ships with Gettext uses `"elixir-format"`. See the [GNU Gettext documentation](https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html#index-msgstr). """ @doc since: "0.19.0" @callback message_format() :: String.t() end ================================================ FILE: lib/gettext/macros.ex ================================================ defmodule Gettext.Macros do @moduledoc """ Macros used by Gettext to provide the gettext family of functions. *Available since v0.26.0.* Macros enable users to use gettext and get **automatic extraction** of translations. See `Gettext` for more information. The macros in this module *that don't end with `_with_backend`* are imported every time you call: use Gettext, backend: MyApp.Gettext ### Explicit backend If you need to use the macros here with an explicit backend and you want extraction to work, you can use the `_with_backend` versions of the macros in this module explicitly instead. defmodule MyApp.Gettext do use Gettext, otp_app: :my_app end defmodule MyApp.Controller do require Gettext.Macros def index(conn, _params) do Gettext.Macros.gettext_with_backend(MyApp.Gettext, "Hello, world!") end end """ @moduledoc since: "0.26.0" alias Gettext.Extractor @doc """ Marks the given message for extraction and returns it unchanged. This macro can be used to mark a message for extraction when `mix gettext.extract` is run. The return value is the given string, so that this macro can be used seamlessly in place of the string to extract. ## Examples dpgettext_noop("errors", "Home page", "Error found!") #=> "Error found!" """ defmacro dpgettext_noop(domain, msgctxt, msgid) do extract_singular_translation(__CALLER__, backend(__CALLER__), domain, msgctxt, msgid) end @doc """ Marks the given message for extraction and returns it unchanged. This macro can be used to mark a message for extraction when `mix gettext.extract` is run. The return value is the given string, so that this macro can be used seamlessly in place of the string to extract. ## Examples dgettext_noop("errors", "Error found!") #=> "Error found!" """ defmacro dgettext_noop(domain, msgid) do extract_singular_translation(__CALLER__, backend(__CALLER__), domain, _msgctxt = nil, msgid) end @doc """ Marks the given message for extraction and returns it unchanged. This macro can be used to mark a message for extraction when `mix gettext.extract` is run. The return value is the given string, so that this macro can be used seamlessly in place of the string to extract. ## Examples gettext_noop("Error found!") #=> "Error found!" """ defmacro gettext_noop(msgid) do extract_singular_translation( __CALLER__, backend(__CALLER__), _domain = :default, _msgctxt = nil, msgid ) end @doc """ Marks the given message for extraction and returns it unchanged. This macro can be used to mark a message for extraction when `mix gettext.extract` is run. The return value is the given string, so that this macro can be used seamlessly in place of the string to extract. ## Examples pgettext_noop("Error found!", "Home page") #=> "Error found!" """ defmacro pgettext_noop(msgid, context) do extract_singular_translation(__CALLER__, backend(__CALLER__), :default, context, msgid) end @doc """ Marks the given message for extraction and returns it unchanged. This macro can be used to mark a message for extraction when `mix gettext.extract` is run. The return value is the given string, so that this macro can be used seamlessly in place of the string to extract. ## Examples dpngettext_noop("errors", "Home page", "Error found!", "Errors found!") #=> "Error found!" """ defmacro dpngettext_noop(domain, msgctxt, msgid, msgid_plural) do extract_plural_translation( __CALLER__, backend(__CALLER__), domain, msgctxt, msgid, msgid_plural ) end @doc """ Marks the given message for extraction and returns `{msgid, msgid_plural}`. This macro can be used to mark a message for extraction when `mix gettext.extract` is run. The return value of this macro is `{msgid, msgid_plural}`. ## Examples my_fun = fn {msgid, msgid_plural} -> # do something with msgid and msgid_plural end my_fun.(dngettext_noop("errors", "One error", "%{count} errors")) """ defmacro dngettext_noop(domain, msgid, msgid_plural) do extract_plural_translation( __CALLER__, backend(__CALLER__), domain, _msgctxt = nil, msgid, msgid_plural ) end @doc """ Marks the given message for extraction and returns it unchanged. This macro can be used to mark a message for extraction when `mix gettext.extract` is run. The return value is the given string, so that this macro can be used seamlessly in place of the string to extract. ## Examples pngettext_noop("Home page", "Error found!", "Errors found!") #=> "Error found!" """ defmacro pngettext_noop(msgctxt, msgid, msgid_plural) do extract_plural_translation( __CALLER__, backend(__CALLER__), _domain = :default, msgctxt, msgid, msgid_plural ) end @doc """ Same as `dngettext_noop("default", msgid, mgsid_plural)`, but will use a per-backend configured default domain if provided. """ defmacro ngettext_noop(msgid, msgid_plural) do extract_plural_translation( __CALLER__, backend(__CALLER__), _domain = :default, _msgctxt = nil, msgid, msgid_plural ) end @doc """ Translates the given `msgid` with a given context (`msgctxt`) in the given `domain`. `bindings` is a map of bindings to support interpolation. See also `Gettext.dpgettext/5`. """ defmacro dpgettext(domain, msgctxt, msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate( __CALLER__, backend(__CALLER__), domain, msgctxt, msgid, bindings ) end @doc """ Translates the given `msgid` in the given `domain`. `bindings` is a map of bindings to support interpolation. See also `Gettext.dgettext/4`. """ defmacro dgettext(domain, msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate( __CALLER__, backend(__CALLER__), domain, _msgctxt = nil, msgid, bindings ) end @doc """ Translates the given `msgid` with the given context (`msgctxt`). `bindings` is a map of bindings to support interpolation. See also `Gettext.pgettext/4`. """ defmacro pgettext(msgctxt, msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate( __CALLER__, backend(__CALLER__), _domain = :default, msgctxt, msgid, bindings ) end @doc """ Same as `dgettext("default", msgid, %{})`, but will use a per-backend configured default domain if provided. See also `Gettext.gettext/3`. """ defmacro gettext(msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate( __CALLER__, backend(__CALLER__), _domain = :default, _msgctxt = nil, msgid, bindings ) end @doc """ Translates the given plural message (`msgid` + `msgid_plural`) with the given context (`msgctxt`) in the given `domain`. `n` is an integer used to determine how to pluralize the message. `bindings` is a map of bindings to support interpolation. See also `Gettext.dpngettext/7`. """ defmacro dpngettext(domain, msgctxt, msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do plural_extract_and_translate( __CALLER__, backend(__CALLER__), domain, msgctxt, msgid, msgid_plural, n, bindings ) end @doc """ Translates the given plural message (`msgid` + `msgid_plural`) in the given `domain`. `n` is an integer used to determine how to pluralize the message. `bindings` is a map of bindings to support interpolation. See also `Gettext.dngettext/6`. """ defmacro dngettext(domain, msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do plural_extract_and_translate( __CALLER__, backend(__CALLER__), domain, _msgctxt = nil, msgid, msgid_plural, n, bindings ) end @doc """ Same as `dngettext("default", msgid, msgid_plural, n, bindings)`, but will use a per-backend configured default domain if provided. See also `Gettext.ngettext/5`. """ defmacro ngettext(msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do plural_extract_and_translate( __CALLER__, backend(__CALLER__), _domain = :default, _msgctxt = nil, msgid, msgid_plural, n, bindings ) end @doc """ Translates the given plural message (`msgid` + `msgid_plural`) with the given context (`msgctxt`). `n` is an integer used to determine how to pluralize the message. `bindings` is a map of bindings to support interpolation. See also `Gettext.pngettext/6`. """ defmacro pngettext(msgctxt, msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do plural_extract_and_translate( __CALLER__, backend(__CALLER__), _domain = :default, msgctxt, msgid, msgid_plural, n, bindings ) end @doc """ Stores an "extracted comment" for the next message. This macro can be used to add comments (Gettext refers to such comments as *extracted comments*) to the next message that will be extracted. Extracted comments will be prefixed with `#.` in POT files. Calling this function multiple times will accumulate the comments; when another Gettext macro (such as `gettext/2`) is called, the comments will be extracted and attached to that message, and they will be flushed so as to start again. This macro always returns `:ok`. ## Examples gettext_comment("The next message is awesome") gettext_comment("Another comment for the next message") gettext("The awesome message") """ defmacro gettext_comment(comment) do comment = expand_to_binary(comment, "comment", __CALLER__) append_extracted_comment(comment) :ok end ## Singular no-op macros (with backend). @doc """ Same as `dpgettext_noop/3`, but takes an explicit backend. """ defmacro dpgettext_noop_with_backend(backend, domain, msgctxt, msgid) do extract_singular_translation(__CALLER__, backend, domain, msgctxt, msgid) end @doc """ Same as `dgettext_noop/2`, but takes an explicit backend. """ defmacro dgettext_noop_with_backend(backend, domain, msgid) do extract_singular_translation(__CALLER__, backend, domain, _msgctxt = nil, msgid) end @doc """ Same as `pgettext_noop/2`, but takes an explicit backend. """ defmacro pgettext_noop_with_backend(backend, msgctxt, msgid) do extract_singular_translation(__CALLER__, backend, _domain = :default, msgctxt, msgid) end @doc """ Same as `gettext_noop/1`, but takes an explicit backend. """ defmacro gettext_noop_with_backend(backend, msgid) do extract_singular_translation(__CALLER__, backend, _domain = :default, _msgctxt = nil, msgid) end ## Plural no-op macros (with backend). @doc """ Same as `dpngettext_noop/4`, but takes an explicit backend. """ defmacro dpngettext_noop_with_backend(backend, domain, msgctxt, msgid, msgid_plural) do extract_plural_translation(__CALLER__, backend, domain, msgctxt, msgid, msgid_plural) end @doc """ Same as `dngettext_noop/3`, but takes an explicit backend. """ defmacro dngettext_noop_with_backend(backend, domain, msgid, msgid_plural) do extract_plural_translation(__CALLER__, backend, domain, _msgctxt = nil, msgid, msgid_plural) end @doc """ Same as `pngettext_noop/3`, but takes an explicit backend. """ defmacro pngettext_noop_with_backend(backend, msgctxt, msgid, msgid_plural) do extract_plural_translation( __CALLER__, backend, _domain = :default, msgctxt, msgid, msgid_plural ) end @doc """ Same as `ngettext_noop/2`, but takes an explicit backend. """ defmacro ngettext_noop_with_backend(backend, msgid, msgid_plural) do extract_plural_translation( __CALLER__, backend, _domain = :default, _msgctxt = nil, msgid, msgid_plural ) end ## Singular macros (with backend). @doc """ Same as `dpgettext/4`, but takes an explicit backend. """ defmacro dpgettext_with_backend(backend, domain, msgctxt, msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate(__CALLER__, backend, domain, msgctxt, msgid, bindings) end @doc """ Same as `dgettext/3`, but takes an explicit backend. """ defmacro dgettext_with_backend(backend, domain, msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate(__CALLER__, backend, domain, _msgctxt = nil, msgid, bindings) end @doc """ Same as `pgettext/3`, but takes an explicit backend. """ defmacro pgettext_with_backend(backend, msgctxt, msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate( __CALLER__, backend, _domain = :default, msgctxt, msgid, bindings ) end @doc """ Same as `gettext/2`, but takes an explicit backend. """ defmacro gettext_with_backend(backend, msgid, bindings \\ Macro.escape(%{})) do singular_extract_and_translate( __CALLER__, backend, _domain = :default, _msgctxt = nil, msgid, bindings ) end @doc """ Same as `dpngettext/6`, but takes an explicit backend. """ defmacro dpngettext_with_backend( backend, domain, msgctxt, msgid, msgid_plural, n, bindings \\ Macro.escape(%{}) ) do plural_extract_and_translate( __CALLER__, backend, domain, msgctxt, msgid, msgid_plural, n, bindings ) end @doc """ Same as `dngettext/5`, but takes an explicit backend. """ defmacro dngettext_with_backend( backend, domain, msgid, msgid_plural, n, bindings \\ Macro.escape(%{}) ) do plural_extract_and_translate( __CALLER__, backend, domain, _msgctxt = nil, msgid, msgid_plural, n, bindings ) end @doc """ Same as `pngettext/5`, but takes an explicit backend. """ defmacro pngettext_with_backend( backend, msgctxt, msgid, msgid_plural, n, bindings \\ Macro.escape(%{}) ) do plural_extract_and_translate( __CALLER__, backend, _domain = :default, msgctxt, msgid, msgid_plural, n, bindings ) end @doc """ Same as `ngettext/4`, but takes an explicit backend. """ defmacro ngettext_with_backend(backend, msgid, msgid_plural, n, bindings \\ Macro.escape(%{})) do plural_extract_and_translate( __CALLER__, backend, _domain = :default, _msgctxt = nil, msgid, msgid_plural, n, bindings ) end ## Helpers defp extract_singular_translation(env, backend, domain, msgctxt, msgid) do backend = expand_backend(backend, env) domain = expand_domain(domain, env) msgid = expand_to_binary(msgid, "msgid", env) msgctxt = expand_to_binary(msgctxt, "msgctxt", env) if Extractor.extracting?() do Extractor.extract( env, backend, domain, msgctxt, msgid, get_and_flush_extracted_comments() ) end msgid end defp extract_plural_translation(env, backend, domain, msgctxt, msgid, msgid_plural) do backend = expand_backend(backend, env) domain = expand_domain(domain, env) msgid = expand_to_binary(msgid, "msgid", env) msgctxt = expand_to_binary(msgctxt, "msgctxt", env) msgid_plural = expand_to_binary(msgid_plural, "msgid_plural", env) if Extractor.extracting?() do Extractor.extract( env, backend, domain, msgctxt, {msgid, msgid_plural}, get_and_flush_extracted_comments() ) end {msgid, msgid_plural} end defp singular_extract_and_translate(env, backend, domain, msgctxt, msgid, bindings) do domain = expand_domain(domain, env) msgid = extract_singular_translation(env, backend, domain, msgctxt, msgid) quote do Gettext.dpgettext( unquote(backend), unquote(domain), unquote(msgctxt), unquote(msgid), unquote(bindings) ) end end defp plural_extract_and_translate( env, backend, domain, msgctxt, msgid, msgid_plural, n, bindings ) do domain = expand_domain(domain, env) {msgid, msgid_plural} = extract_plural_translation(env, backend, domain, msgctxt, msgid, msgid_plural) quote do Gettext.dpngettext( unquote(backend), unquote(domain), unquote(msgctxt), unquote(msgid), unquote(msgid_plural), unquote(n), unquote(bindings) ) end end defp expand_domain(:default, _env), do: :default defp expand_domain(domain, env), do: expand_to_binary(domain, "domain", env) defp backend(%Macro.Env{} = env) do Module.get_attribute(env.module, :__gettext_backend__) || raise """ in order to use Gettext.Macros, you must: use Gettext, backend: ... """ end defp expand_to_binary(term, what, %Macro.Env{} = env) when what in ~w(domain msgctxt msgid msgid_plural comment) do raiser = fn term -> gettext_module = Module.get_attribute(env.module, :__gettext_backend__) raise ArgumentError, """ Gettext macros expect message keys (msgid and msgid_plural), domains, and comments to expand to strings at compile-time, but the given #{what} doesn't. This is what the macro received: #{inspect(term)} Dynamic messages should be avoided as they limit Gettext's ability to extract messages from your source code. If you are sure you need dynamic lookup, you can use the functions in the Gettext module: string = "hello world" Gettext.gettext(#{if(gettext_module, do: inspect(gettext_module), else: "backend")}, string) """ end validated_expand_to_binary(term, env, raiser) end defp validated_expand_to_binary({:<>, _, pieces}, env, raiser) do Enum.map_join(pieces, &validated_expand_to_binary(&1, env, raiser)) end defp validated_expand_to_binary({:<<>>, _, pieces}, env, raiser) do Enum.map_join(pieces, &validated_expand_to_binary(&1, env, raiser)) end # We support nil too in order to fall back to a nil context and always use the *p # variants of the Gettext macros. defp validated_expand_to_binary(term, _env, _raiser) when is_binary(term) or is_nil(term) do term end defp validated_expand_to_binary(term, env, raiser) do case Macro.expand(term, env) do term when is_binary(term) or is_nil(term) -> term other -> raiser.(other) end end defp expand_backend(term, %Macro.Env{} = env) do case Macro.expand(term, env) do term when is_atom(term) and term not in [nil, false, true] -> term _other -> raise ArgumentError, """ Gettext.Macros macros (that end with "_with_backend") expect the backend argument to be an atom at compile-time, but the given term doesn't. This is what the macro received: #{inspect(term)} Dynamic messages should be avoided as they limit Gettext's ability to extract messages from your source code. If you are sure you need dynamic lookup, you can use the functions in the Gettext module: string = "hello world" Gettext.gettext(backend, string) """ end end defp append_extracted_comment(comment) do existing = Process.get(:gettext_comments, []) Process.put(:gettext_comments, [" " <> comment | existing]) :ok end defp get_and_flush_extracted_comments() do Enum.reverse(Process.delete(:gettext_comments) || []) end end ================================================ FILE: lib/gettext/merger.ex ================================================ defmodule Gettext.Merger do @moduledoc false alias Expo.PO alias Expo.Message alias Expo.Messages alias Gettext.Fuzzy alias Gettext.Plural @new_po_informative_comment """ # "msgid"s in this file come from POT (.pot) files. ## ## Do not add, change, or remove "msgid"s manually here as ## they're tied to the ones in the corresponding POT file ## (with the same domain). ## ## Use "mix gettext.extract --merge" or "mix gettext.merge" ## to merge POT files into PO files. """ @doc """ Merges two `Gettext.PO` structs representing a PO file and an updated POT (or PO) file into a new `Gettext.PO` struct. `old` is an existing PO file (that contains messages) which will be "updated" with the messages in the `new` POT or PO file. messages in `old` will kept as long as they match with messages in `new`; all other messages will be discarded (as `new` is considered to be the reference). The `Gettext.PO` struct that this function returns is *always* meant to be a PO file, not a POT file. `new` can be: * a POT file (usually created or updated by the `mix gettext.extract` task) or * a newly created PO file with up-to-date source references (but old messages) Note that all translator comments in `new` will be discarded in favour of the ones in `old`. Reference comments and extracted comments will be taken from `new` instead. The following rules are observed: * matching messages are merged as follows: * existing msgstr are preserved (the ones in the POT file are empty anyways) * existing translator comments are preserved (there are no translator comments in POT files) * existing extracted comments are replaced by new extracted comments * existing references are discarded (as they're now outdated) and replaced by the references in the POT file """ @spec merge(Messages.t(), Messages.t(), String.t(), Keyword.t(), Keyword.t()) :: {Messages.t(), map()} def merge(%Messages{} = old, %Messages{} = new, locale, opts, gettext_config) when is_binary(locale) and is_list(opts) do opts = put_plural_forms_opt(opts, old, locale) stats = %{new: 0, exact_matches: 0, fuzzy_matches: 0, removed: 0, marked_as_obsolete: 0} {messages, stats} = merge_messages(old.messages, new.messages, opts, gettext_config, stats) po = %Messages{ top_comments: old.top_comments, headers: old.headers, file: old.file, messages: messages } {po, stats} end defp merge_messages(old, new, opts, gettext_config, stats) do fuzzy? = Keyword.fetch!(opts, :fuzzy) fuzzy_threshold = Keyword.fetch!(opts, :fuzzy_threshold) plural_forms = Keyword.fetch!(opts, :plural_forms) custom_flags_to_keep = Keyword.get(gettext_config, :custom_flags_to_keep, []) old = Map.new(old, &{Message.key(&1), &1}) {messages, {stats, unused}} = Enum.map_reduce(new, {stats, _unused = old}, fn message, {stats_acc, unused} -> key = Message.key(message) message = adjust_number_of_plural_forms(message, plural_forms) case Map.fetch(old, key) do {:ok, exact_match} -> stats = update_in(stats_acc.exact_matches, &(&1 + 1)) {merge_two_messages(exact_match, message, custom_flags_to_keep), {stats, Map.delete(unused, key)}} :error when fuzzy? -> case maybe_merge_fuzzy(message, old, key, fuzzy_threshold) do {:matched, match, fuzzy_merged} -> stats_acc = update_in(stats_acc.fuzzy_matches, &(&1 + 1)) unused = Map.delete(unused, Message.key(match)) fuzzy_merged = if Keyword.get(opts, :store_previous_message_on_fuzzy_match, false) do Map.update!(fuzzy_merged, :previous_messages, fn previous -> Enum.uniq_by(previous ++ [match], &Message.key/1) end) else fuzzy_merged end {fuzzy_merged, {stats_acc, unused}} :nomatch -> stats_acc = update_in(stats_acc.new, &(&1 + 1)) {message, {stats_acc, unused}} end :error -> stats_acc = update_in(stats_acc.new, &(&1 + 1)) {message, {stats_acc, unused}} end end) messages = Enum.map(messages, &%{&1 | obsolete: false}) {messages, stats} = case Keyword.get(opts, :on_obsolete, :delete) do :mark_as_obsolete -> {messages ++ (unused |> Map.values() |> Enum.map(&%{&1 | obsolete: true})), put_in(stats.marked_as_obsolete, map_size(unused))} :delete -> {messages, put_in(stats.removed, map_size(unused))} end {messages, stats} end defp adjust_number_of_plural_forms(%Message.Plural{} = message, plural_forms) when plural_forms > 0 do new_msgstr = Map.new(0..(plural_forms - 1), &{&1, [""]}) %{message | msgstr: new_msgstr} end defp adjust_number_of_plural_forms(%Message.Singular{} = message, _plural_forms) do message end defp maybe_merge_fuzzy(message, old, key, fuzzy_threshold) do if matched = find_fuzzy_match(old, key, fuzzy_threshold) do {:matched, matched, Fuzzy.merge(message, matched)} else :nomatch end end defp find_fuzzy_match(messages, key, threshold) do matcher = Fuzzy.matcher(threshold) candidates = for {k, message} <- messages, match = matcher.(k, key), match != :nomatch, do: {message, match} if candidates == [] do nil else {message, _match} = Enum.max_by(candidates, fn {_t, {:match, distance}} -> distance end) message end end # msgid, msgid_plural: they're the same # msgctxt: it's the same, even if it's not present (nil) # msgstr: new.msgstr should be empty since it comes from a POT file # comments: new has no translator comments as it comes from POT # extracted_comments: we should take the new most recent ones # flags: we should take the new flags and preserve the fuzzy flag # references: new contains the updated and most recent references defp merge_two_messages(old, new, custom_flags_to_keep) do old |> Message.merge(new) |> Map.merge(%{ comments: old.comments, extracted_comments: new.extracted_comments, flags: merge_flags(old, new, custom_flags_to_keep), references: new.references }) end defp merge_flags(old_message, new_message, custom_flags_to_keep) do # Force the "fuzzy" flag. flags_to_keep = Enum.uniq(["fuzzy" | custom_flags_to_keep]) %{flags: flags} = Enum.reduce(flags_to_keep, new_message, fn flag, message -> if Message.has_flag?(old_message, flag) do Message.append_flag(message, flag) else message end end) flags end @doc """ Returns the contents of a new PO file to be written at `po_file` from the POT template in `pot_file`. The new PO file will have: * the `Language` header set based on the locale (extracted from the path) * the messages of the POT file (no merging is needed as there are no messages in the PO file) Comments in `pot_file` that start with `##` will be discarded and not copied over the new PO file as they're meant to be comments generated by tools or comments directed to developers. """ def new_po_file(po_file, pot_file, locale, opts) when is_binary(locale) and is_list(opts) do pot = PO.parse_file!(pot_file) opts = put_plural_forms_opt(opts, pot, locale) plural_forms = Keyword.fetch!(opts, :plural_forms) plural_forms_header = Keyword.fetch!(opts, :plural_forms_header) po = %Messages{ top_comments: String.split(@new_po_informative_comment, "\n", trim: true), headers: headers_for_new_po_file(locale, plural_forms_header), file: po_file, messages: Enum.map(pot.messages, &prepare_new_message(&1, plural_forms)) } stats = %{ new: length(po.messages), exact_matches: 0, fuzzy_matches: 0, removed: 0, marked_as_obsolete: 0 } {po, stats} end @doc false @spec prune_references(messages :: Messages.t(), gettext_config :: Keyword.t()) :: Messages.t() def prune_references(%Messages{} = all, gettext_config) when is_list(gettext_config) do cond do # Empty out all references. not Keyword.get(gettext_config, :write_reference_comments, true) -> put_in(all, [Access.key!(:messages), Access.all(), Access.key(:references)], []) # Remove lines from references and unique them. not Keyword.get(gettext_config, :write_reference_line_numbers, true) -> update_in( all, [Access.key!(:messages), Access.all(), Access.key(:references)], &remove_line_and_unique_references/1 ) true -> all end end defp remove_line_and_unique_references(references) do {unique_refs, _} = references |> update_in([Access.all(), Access.all()], fn {file, _line} -> file file -> file end) |> Enum.map_reduce(MapSet.new(), fn line, existing_references -> unique_line = Enum.uniq(line) -- MapSet.to_list(existing_references) {unique_line, MapSet.union(existing_references, MapSet.new(unique_line))} end) Enum.reject(unique_refs, &match?([], &1)) end defp headers_for_new_po_file(locale, plural_forms_header) do [ "", ~s(Language: #{locale}\n), ~s(Plural-Forms: #{plural_forms_header}\n) ] end defp prepare_new_message(message, plural_forms) do message |> strip_double_hash_comments() |> adjust_number_of_plural_forms(plural_forms) end defp strip_double_hash_comments(%{comments: comments} = message) do %{message | comments: Enum.reject(comments, &match?("#" <> _, &1))} end # TODO: simplify code here once we remove support for :plural_forms. defp put_plural_forms_opt(opts, messages, locale) do plural_mod = Application.get_env(:gettext, :plural_forms, Gettext.Plural) default_nplurals = plural_mod.nplurals(Plural.plural_info(locale, messages, plural_mod)) opts = Keyword.put_new(opts, :plural_forms, default_nplurals) Keyword.put_new_lazy(opts, :plural_forms_header, fn -> requested_nplurals = Keyword.fetch!(opts, :plural_forms) # If nplurals is overridden to a non-default value by the user the # implementation will not be able to provide a correct header therefore # the header is just set to `nplurals=#{n}` and it is up to the user to # put a complete plural forms header themselves. if requested_nplurals == default_nplurals do Plural.plural_forms_header_impl(locale, messages, plural_mod) else "nplurals=#{requested_nplurals}" end end) end end ================================================ FILE: lib/gettext/missing_bindings_error.ex ================================================ defmodule Gettext.MissingBindingsError do @moduledoc """ An error message raised for missing bindings errors. """ @enforce_keys [:backend, :domain, :msgctxt, :locale, :msgid, :missing] defexception [:backend, :domain, :msgctxt, :locale, :msgid, :missing] @type t() :: %__MODULE__{} @impl true def message(%__MODULE__{ backend: backend, domain: domain, msgctxt: msgctxt, locale: locale, msgid: msgid, missing: missing }) do "missing Gettext bindings: #{inspect(missing)} (backend #{inspect(backend)}, " <> "locale #{inspect(locale)}, domain #{inspect(domain)}, msgctxt #{inspect(msgctxt)}, " <> "msgid #{inspect(msgid)})" end end ================================================ FILE: lib/gettext/plural.ex ================================================ defmodule Gettext.Plural do @moduledoc """ Behaviour and default implementation for finding plural forms in given locales. This module both defines the `Gettext.Plural` behaviour and provides a default implementation for it. ## Plural Forms > For a given language, there is a grammatical rule on how to change words > depending on the number qualifying the word. Different languages can have > different rules. [[source]](https://udn.realityripple.com/docs/Mozilla/Localization/Localization_and_Plurals) Such grammatical rules define a number of **plural forms**. For example, English has two plural forms: one for when there is just one element (the *singular*) and another one for when there are zero or more than one elements (the *plural*). There are languages which only have one plural form and there are languages which have more than two. In GNU Gettext (and in Gettext for Elixir), plural forms are represented by increasing 0-indexed integers. For example, in English `0` means singular and `1` means plural. The goal of this module is to determine, given a locale: * how many plural forms exist in that locale (`nplurals/1`); * to what plural form a given number of elements belongs to in that locale (`plural/2`). ## Default Implementation `Gettext.Plural` provides a default implementation of a plural module. Most common languages used on Earth should be covered by this default implementation. If custom pluralization rules are needed (for example, to add additional languages) a different plural module can be specified when creating a Gettext backend. For example, pluralization rules for the Elvish language could be added as follows: defmodule MyApp.Plural do @behaviour Gettext.Plural def nplurals("elv"), do: 3 def plural("elv", 0), do: 0 def plural("elv", 1), do: 1 def plural("elv", _), do: 2 # Fall back to Gettext.Plural defdelegate nplurals(locale), to: Gettext.Plural defdelegate plural(locale, n), to: Gettext.Plural end The mathematical expressions used in this module to determine the plural form of a given number of elements are taken from [this page](http://localization-guide.readthedocs.org/en/latest/l10n/pluralforms.html#f2) as well as from [Mozilla's guide on "Localization and plurals"](https://udn.realityripple.com/docs/Mozilla/Localization/Localization_and_Plurals). ## Changing Implementations Once you have defined your custom plural forms module, you can use it in two ways. You can set it for all Gettext backends in your configuration: # For example, in config/config.exs config :gettext, :plural_forms, MyApp.Plural or you can set it for each specific backend when you call `use Gettext`: defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app, plural_forms: MyApp.Plural end > #### Compile-time Configuration {: .warning} > > Set `:plural_forms` in your `config/config.exs` and > not in `config/runtime.exs`, as Gettext reads this option when > compiling your backends. Task such as `mix gettext.merge` use the plural backend configured under the `:gettext` application, so in general the global configuration approach is preferred. Some tasks also allow the number of plural forms to be given explicitly, for example: mix gettext.merge priv/gettext --locale=gsw_CH --plural-forms=2 ## Unknown Locales Trying to call `Gettext.Plural` functions with unknown locales will result in a `Gettext.Plural.UnknownLocaleError` exception. ## Language and Territory Often, a locale is composed as a language and territory pair, such as `en_US`. The default implementation for `Gettext.Plural` handles `xx_YY` by forwarding it to `xx` (except for *just Brazilian Portuguese*, `pt_BR`, which is not forwarded to `pt` as pluralization rules differ slightly). We treat the underscore as a separator according to [ISO 15897](https://en.wikipedia.org/wiki/ISO/IEC_15897). Sometimes, a dash `-` is used as a separator (for example [BCP47](https://en.wikipedia.org/wiki/IETF_language_tag) locales use this as in `en-US`): this is *not forwarded* to `en` in the default `Gettext.Plural` (and it will raise an `Gettext.Plural.UnknownLocaleError` exception if there are no messages for `en-US`). We recommend defining a custom plural forms module that replaces `-` with `_` if needed. ## Examples An example of the plural form of a given number of elements in the Polish language: iex> Gettext.Plural.plural("pl", 1) 0 iex> Gettext.Plural.plural("pl", 2) 1 iex> Gettext.Plural.plural("pl", 5) 2 iex> Gettext.Plural.plural("pl", 112) 2 As expected, `nplurals/1` returns the possible number of plural forms: iex> Gettext.Plural.nplurals("pl") 3 """ alias Expo.Messages # Types @typedoc """ A locale passed to `c:plural/2`. """ @typedoc since: "0.22.0" @type locale() :: String.t() @typedoc """ The context passed to the optional `c:init/1` callback. If `:plural_forms_header` is present, it contains the contents of the `Plural-Forms` Gettext header. """ @typedoc since: "0.22.0" @type pluralization_context() :: %{ required(:locale) => locale(), optional(:plural_forms_header) => String.t() } @typedoc """ The term that the optional `c:init/1` callback returns. """ @typedoc since: "0.22.0" @type plural_info() :: term() ## Behaviour definition @doc """ Should initialize the context for `c:nplurals/1` and `c:plural/2`. This callback should perform all preparations for the provided locale, which is part of the pluralization context (see `t:pluralization_context/0`). For example, you can use this callback to parse the `Plural-Forms` header and determine pluralization rules for the locale. If defined, Gettext calls this callback *once* at compile time. If not defined, the returned `plural_info` will be equals to the locale found in `pluralization_context`. ## Examples defmodule MyApp.Plural do @behaviour Gettext.Plural @impl true def init(%{locale: _locale, plural_forms_header: header}) do {nplurals, rule} = parse_plural_forms_header(header) # This is what other callbacks can use to determine the plural. {nplurals, rule} end @impl true def nplurals({_locale, nplurals, _rule}), do: nplurals # ... end """ @doc since: "0.22.0" @callback init(pluralization_context()) :: plural_info() @doc """ Should return the number of possible plural forms in the given `locale`. """ @callback nplurals(plural_info()) :: pos_integer() @doc """ Should return the plural form in the given `locale` for the given `count` of elements. """ @callback plural(plural_info(), count :: integer()) :: plural_form :: non_neg_integer() @doc """ Should return the value of the `Plural-Forms` header for the given `locale`, if present. If the value of the `Plural-Forms` header is unavailable for any reason, this function should return `nil`. This callback is optional. If it's not defined, the fallback returns: "nplurals={nplurals};" """ @doc since: "0.22.0" @callback plural_forms_header(locale()) :: String.t() | nil @optional_callbacks init: 1, plural_forms_header: 1 defmodule UnknownLocaleError do @moduledoc """ Raised when a pluralized module doesn't know how to handle a locale. ## Examples raise Gettext.Plural.UnknownLocaleError, "en-US" """ defexception [:message] def exception(locale) when is_binary(locale) do message = """ unknown locale #{inspect(locale)}. If this is a locale you need to handle, consider using a custom pluralizer module instead of the default Gettext.Plural. You can read more about this on the Gettext docs at https://hexdocs.pm/gettext/Gettext.Plural.html """ %__MODULE__{message: message} end end # Behaviour implementation. # Default implementation of the init/1 callback, in case the user uses # Gettext.Plural as their plural forms module. @doc false def init(context) def init(%{locale: locale, plural_forms_header: plural_forms_header}) do case Expo.PluralForms.parse(plural_forms_header) do {:ok, plural_forms} -> {locale, plural_forms} {:error, _reason} -> message_about_header = case Expo.PluralForms.plural_form(locale) do {:ok, plural_form} -> """ For the #{inspect(locale)} locale, you can use the following header: #{Expo.PluralForms.to_string(plural_form)} """ :error -> "" end # Fall back to parsing headers such as "nplurals=3", without the "plural=..." part. # TODO: remove this in v0.24.0 with "nplurals=" <> rest <- String.trim(plural_forms_header), {plural_forms, _rest} <- Integer.parse(rest) do IO.warn(""" Plural-Forms headers in the form "nplurals=" (without the "plural=" part \ following) are invalid and support for them will be removed in future Gettext \ versions. Make sure to use a complete Plural-Forms header, which also specifies \ the pluralization rules, or remove the Plural-Forms header completely. If you \ do the latter, Gettext will use its built-in pluralization rules for the languages \ it knows about (see Gettext.Plural).#{message_about_header}\ """) {locale, plural_forms} else _other -> locale end end end def init(%{locale: locale}), do: locale # Number of plural forms. @doc """ Default implementation of the `c:nplurals/1` callback. """ def nplurals(locale) # TODO: this is a fallback for headers such as "nplurals=x", without "plural=...". # We should remove support for these at some point. def nplurals({_locale, nplurals}) when is_integer(nplurals) do nplurals end # If the nplurals was provided, we don't need to look at the locale. def nplurals({_locale, plural_forms}) do plural_forms.nplurals end def nplurals(locale) do case Expo.PluralForms.plural_form(locale) do {:ok, plural_form} -> plural_form.nplurals :error -> recall_if_territory_or_raise(locale, &nplurals/1) end end @doc """ Default implementation of the `c:plural/2` callback. """ def plural(locale, count) # TODO: this is a fallback for headers such as "nplurals=x", without "plural=...". # We should remove support for these at some point. def plural({locale, nplurals}, count) when is_integer(nplurals) do plural(locale, count) end def plural({_locale, plural_form}, count) do Expo.PluralForms.index(plural_form, count) end def plural(locale, count) do case Expo.PluralForms.plural_form(locale) do {:ok, plural_form} -> Expo.PluralForms.index(plural_form, count) :error -> recall_if_territory_or_raise(locale, &plural(&1, count)) end end def plural_forms_header(locale) do case Expo.PluralForms.plural_form(locale) do {:ok, plural_form} -> Expo.PluralForms.to_string(plural_form) :error -> recall_if_territory_or_raise(locale, &plural_forms_header(&1)) end rescue UnknownLocaleError -> nil end defp recall_if_territory_or_raise(locale, fun) do case String.split(locale, "_", parts: 2, trim: true) do [lang, _territory] -> fun.(lang) _other -> raise UnknownLocaleError, locale end end @doc false def plural_info(locale, messages_struct, plural_mod) do Code.ensure_compiled!(plural_mod) if function_exported?(plural_mod, :init, 1) do pluralization_context = case IO.iodata_to_binary(Messages.get_header(messages_struct, "Plural-Forms")) do "" -> %{locale: locale} plural_forms -> %{locale: locale, plural_forms_header: plural_forms} end plural_mod.init(pluralization_context) else locale end end @doc false def plural_forms_header_impl(locale, messages_struct, plural_mod) do Code.ensure_compiled!(plural_mod) plural_forms_header = if function_exported?(plural_mod, :plural_forms_header, 1) do plural_mod.plural_forms_header(locale) end if plural_forms_header do plural_forms_header else nplurals = plural_mod.nplurals(plural_info(locale, messages_struct, plural_mod)) "nplurals=#{nplurals}" end end end ================================================ FILE: lib/gettext/plural_form_error.ex ================================================ defmodule Gettext.PluralFormError do @moduledoc """ An generic error for when a plural form is missing for a given locale. """ @enforce_keys [:form, :locale, :file, :line] defexception [:form, :locale, :file, :line] @type t() :: %__MODULE__{ form: non_neg_integer(), locale: String.t(), file: String.t(), line: pos_integer() } @impl true def message(%__MODULE__{form: form, locale: locale, file: file, line: line}) do "plural form #{form} is required for locale #{inspect(locale)} " <> "but is missing for message compiled from #{file}:#{line}" end end ================================================ FILE: lib/gettext.ex ================================================ defmodule Gettext do @moduledoc ~S""" The `Gettext` module provides a [gettext](https://www.gnu.org/software/gettext/)-based API for working with internationalized applications. ## Basic Overview When you use Gettext, you replace hardcoded user-facing text like this: "Hello world" with calls like this: gettext("Hello world") Here, the string `"Hello world"` serves two purposes: 1. It's displayed by default (if no translation is specified in the current language). This means that, at the very least, switching from a hardcoded string to a Gettext call is harmless. 2. It serves as the **message ID** to which translations will be mapped. An example translation workflow is as follows. First, call `mix gettext.extract` to extract `gettext()` calls to `.pot` ([Portable Object Template](https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html)) files, which are the base for all translations. These files are *templates*, which means they only contain message IDs, and not actual translated strings. POT files have entries like this: #: lib/my_app_web/live/hello_live.html.heex:2 #, elixir-autogen, elixir-format msgid "Hello world" msgstr "" Then, call `mix gettext.merge priv/gettext` to update all locale-specific `.po` (Portable Object) files so that they include this message ID. Entries in PO files contain translations for their specific locale. For example, in a PO file for Italian, the entry above would look like this: #: lib/my_app_web/live/hello_live.html.heex:2 #, elixir-autogen, elixir-format msgid "Hello world" msgstr "Ciao mondo" The English string is the `msgid` which is used to look up the correct Italian string. That's handy, because unlike a generic key like `site.greeting` (as some translations systems use), the message ID tells exactly what needs to be translated. This is easier to work with for translators, for example. But it raises a question: what if you change the original English string in the code? Does that break all translations, requiring manual edits everywhere? Not necessarily. After you run `mix gettext.extract` again, the next `mix gettext.merge` can do **fuzzy matching**. So, if you change `"Hello world"` to `"Hello world!"`, Gettext will see that the new message ID is similar to an existing `msgid`, and will do two things: 1. It will update the `msgid` in all `.po` files to match the new text. 2. It will mark those entries as "fuzzy"; this hints that a (probably human) translator should check whether the Italian translation of this string needs an update. The resulting change in the `.po` file is this (note the "fuzzy" annotation): #: lib/myapp_web/live/hello_live.html.heex:2 #, elixir-autogen, elixir-format, fuzzy msgid "Hello world!" msgstr "Ciao mondo" This "fuzzy matching" behavior can be configured or disabled, but its existence makes updating translations to match changes in the base text easier. The rest of the documentation will cover the Gettext API in detail. ## Gettext API To use Gettext, you will need a **backend module** which stores and retrieves translations from PO files. You can create such a module by using `Gettext.Backend`: defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Now, you can import all the necessary translation macros (defined in `Gettext.Macros`) into any module by using `Gettext`: defmodule MyApp.SomeModule do use Gettext, backend: MyApp.Gettext def showcase_gettext do # Simple message gettext("Hello world") # Plural message ngettext( "Here is the string to translate", "Here are the strings to translate", 3 ) # Domain-based message dgettext("errors", "Here is the error message to translate") # Context-based message pgettext("email", "Email text to translate") end end The arguments for the Gettext macros and their order can be derived from their names. For example, for [`dpgettext/4`](`Gettext.Macros.dpgettext/4`) the arguments are: `domain`, `context`, `msgid`, `bindings` (default to `%{}`). Messages are looked up from `.po` files. In the following sections we will explore exactly what are those files before we explore the "Gettext API" in detail. > #### Recent Updates {: .info} > > Before v0.26.0 of this library, the workflow described in this section > was slightly different. Check out [the > changelog](https://github.com/elixir-gettext/gettext/blob/main/CHANGELOG.md) for more > details, but the gist is that `use Gettext` used to define macros in the calling module. > This created heavy compile-time dependencies which would cause slow recompilation > in larger applications. ## Messages Messages are stored inside PO (Portable Object) files, with a `.po` extension. For example, this is a snippet from a PO file: # This is a comment msgid "Hello world!" msgstr "Ciao mondo!" PO files containing messages for an application must be stored in a directory (by default it's `priv/gettext`) that has the following structure: gettext directory └─ locale └─ LC_MESSAGES ├─ domain_1.po ├─ domain_2.po └─ domain_3.po Here, `locale` is the locale of the messages (for example, `en_US`), `LC_MESSAGES` is a fixed directory, and `domain_i.po` are PO files containing domain-scoped messages. For more information on domains, check out the "Domains" section below. A concrete example of such a directory structure could look like this: priv/gettext └─ en_US | └─ LC_MESSAGES | ├─ default.po | └─ errors.po └─ it └─ LC_MESSAGES ├─ default.po └─ errors.po By default, Gettext expects messages to be stored under the `priv/gettext` directory of an application. This behaviour can be changed by specifying a `:priv` option when using `Gettext`: # Look for messages in my_app/priv/messages instead of # my_app/priv/gettext use Gettext.Backend, otp_app: :my_app, priv: "priv/messages" The messages directory specified by the `:priv` option should be a directory inside `priv/`, otherwise some things won't work as expected. ## Locale At runtime, all gettext-related functions and macros that do not explicitly take a locale as an argument read the locale from the backend and fall back to Gettext's default locale. `Gettext.put_locale/1` can be used to change the locale of all backends for the current Elixir process. That's the preferred mechanism for setting the locale at runtime. `Gettext.put_locale/2` can be used when you want to set the locale of one specific Gettext backend without affecting other Gettext backends. Similarly, `Gettext.get_locale/0` gets the locale for all backends in the current process. `Gettext.get_locale/1` gets the locale of a specific backend for the current process. Check their documentation for more information. Locales are expressed as strings (like `"en"` or `"fr"`); they can be arbitrary strings as long as they match a directory name. As mentioned above, the locale is stored **per-process** (in the process dictionary): this means that the locale must be set in every new process in order to have the right locale available for that process. Pay attention to this behaviour, since not setting the locale *will not* result in any errors when `Gettext.get_locale/0` or `Gettext.get_locale/1` are called; the default locale will be returned instead. To decide which locale to use, each gettext-related function in a given backend follows these steps: * if there is a backend-specific locale for the given backend for this process (see `put_locale/2`), use that, otherwise * if there is a global locale for this process (see `put_locale/1`), use that, otherwise * if there is a backend-specific default locale in the configuration for that backend's `:otp_app` (see the "Default locale" section below), use that, otherwise * use the default global Gettext locale (see the "Default locale" section below) ### Default locale The global Gettext default locale can be configured through the `:default_locale` key of the `:gettext` application: config :gettext, :default_locale, "fr" By default the global locale is `"en"`. See also `get_locale/0` and `put_locale/1`. If for some reason a backend requires a different `:default_locale` than all other backends, you can set the `:default_locale` inside the backend configuration, but this approach is generally discouraged as it makes it hard to track which locale each backend is using: config :my_app, MyApp.Gettext, default_locale: "fr" ## Gettext API There are two ways to use Gettext: * using macros from your own Gettext module, like `MyApp.Gettext` * using functions from the `Gettext` module These two approaches are different and each one has its own use case. ### Using macros Each module that calls `use Gettext.Backend` is usually referred to as a "Gettext backend", as it implements the `Gettext.Backend` behaviour. When a module then calls `use Gettext, backend: MyApp.Gettext`, all the macros defined in `Gettext.Macros` are imported into that module, such as: * [`gettext/2`](`Gettext.Macros.gettext/2`) * [`dgettext/3`](`Gettext.Macros.dgettext/3`) * [`pgettext/3`](`Gettext.Macros.pgettext/3`) Using macros is preferred as Gettext is able to automatically sync the messages in your code with PO files. This, however, imposes a constraint: arguments passed to any of these macros have to be strings **at compile time**. This means that they have to be string literals or something that expands to a string literal at compile time (for example, a module attribute like `@my_string "foo"`). These are all valid uses of the Gettext macros: Gettext.put_locale(MyApp.Gettext, "it") use Gettext, backend: MyApp.Gettext gettext("Hello world") #=> "Ciao mondo" @msgid "Hello world" gettext(@msgid) #=> "Ciao mondo" The `*gettext` macros raise an `ArgumentError` exception if they receive a `domain`, `msgctxt`, `msgid`, or `msgid_plural` that doesn't expand to a string *at compile time*: msgid = "Hello world" gettext(msgid) #=> ** (ArgumentError) msgid must be a string literal Using compile-time strings isn't always possible. For this reason, the `Gettext` module provides a set of functions as well. ### Using functions If compile-time strings cannot be used, the solution is to use the functions in the `Gettext` module instead of the macros described above. These functions perfectly mirror the macro API, but they all expect a Gettext backend module as the first argument. defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Gettext.put_locale(MyApp.Gettext, "pt_BR") msgid = "Hello world" Gettext.gettext(MyApp.Gettext, msgid) #=> "Olá mundo" While using functions from the `Gettext` module yields the same results as using macros (with the added benefit of dynamic arguments), all the compile-time features mentioned in the previous section are lost. ## Domains The [`dgettext`](`Gettext.Macros.dgettext/3`) and [`dngettext`](`Gettext.Macros.dngettext/5`) macros (and their function counterparts) also accept a *domain* as one of the arguments. The domain of a message is determined by the name of the PO file that contains that message. For example, the domain of messages in the `it/LC_MESSAGES/errors.po` file is `"errors"`, so those messages would need to be retrieved with `dgettext` or `dngettext`: dgettext("errors", "Error!") #=> "Errore!" When backend `gettext`, `ngettext`, or `pgettext` are used, the backend's default domain is used (which defaults to "default"). The `Gettext` functions accepting a backend (`gettext/3`, `ngettext/5`, and `pgettext/4`) _always_ use a domain of "default". ### Default Domain Each backend can be configured with a specific `:default_domain` that replaces `"default"` in `gettext/2`, `pgettext/3`, and `ngettext/4` for that backend. defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app, default_domain: "messages" end config :my_app, MyApp.Gettext, default_domain: "messages" ## Contexts The GNU Gettext implementation supports [*contexts*](https://www.gnu.org/software/gettext/manual/html_node/Contexts.html), which are a way to contextualize messages. For example, in English, the word "file" could be used both as a noun as well as a verb. Contexts can be used to solve similar problems: you could have a `imperative_verbs` context and a `nouns` context as to avoid ambiguity. The functions that handle contexts have a `p` in their name (to match the GNU Gettext API), and are `pgettext`, `dpgettext`, `pngettext`, and `dpngettext`. The "p" stands for "particular". ## Interpolation All `*gettext` functions and macros provided by Gettext support interpolation. Interpolation keys can be placed in `msgid`s or `msgid_plural`s with by enclosing them in `%{` and `}`, like this: "This is an %{interpolated} string" Interpolation bindings can be passed as an argument to all of the `*gettext` functions/macros. For example, given the following PO file for the `"it"` locale: msgid "Hello, %{name}!" msgstr "Ciao, %{name}!" interpolation can be done like follows: Gettext.put_locale(MyApp.Gettext, "it") gettext("Hello, %{name}!", name: "Meg") #=> "Ciao, Meg!" Interpolation keys that are in a string but not in the provided bindings result in an exception: gettext("Hello, %{name}!") #=> ** (Gettext.MissingBindingsError) ... Keys that are in the interpolation bindings but that don't occur in the string are ignored. Interpolations in Gettext are often expanded at compile time, ensuring a low performance cost when running them at runtime. ## Pluralization Pluralization in Gettext for Elixir works very similar to how pluralization works in GNU Gettext. The `*ngettext` functions/macros accept a `msgid`, a `msgid_plural`, and a count of elements; the right message is chosen based on the **pluralization rule** for the given locale. For example, given the following snippet of PO file for the `"it"` locale: msgid "One error" msgid_plural "%{count} errors" msgstr[0] "Un errore" msgstr[1] "%{count} errori" the `ngettext` macro can be used like this: Gettext.put_locale(MyApp.Gettext, "it") ngettext("One error", "%{count} errors", 3) #=> "3 errori" The `%{count}` interpolation key is a special key since it gets replaced by the number of elements argument passed to `*ngettext`, like if the `count: 3` key-value pair were in the interpolation bindings. Hence, never pass the `count` key in the bindings: # `count: 4` is ignored here ngettext("One error", "%{count} errors", 3, count: 4) #=> "3 errori" You can specify a "pluralizer" module via the `:plural_forms` option in the configuration for each Gettext backend. defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app, plural_forms: MyApp.PluralForms end To learn more about pluralization rules, plural forms and what they mean to Gettext check the documentation for `Gettext.Plural`. ## Missing messages When a message is missing in the specified locale (both with functions and with macros), the argument is returned: * in case of calls to `gettext`/`dgettext`/`pgettext`/`dpgettext`, the `msgid` argument is returned as is; * in case of calls to `ngettext`/`dngettext`/`pngettext`/`dpngettext`, the `msgid` argument is returned in case of a singular value and the `msgid_plural` is returned in case of a plural value (following the English pluralization rule). For example: Gettext.put_locale(MyApp.Gettext, "foo") gettext("Hey there") #=> "Hey there" ngettext("One error", "%{count} errors", 3) #=> "3 errors" ### Empty messages When a `msgstr` is empty (`""`), the message is considered missing and the behaviour described above for missing message is applied. A plural message is considered to have an empty `msgstr` if at least one message in the `msgstr` is empty. ## Compile-time features As mentioned above, using the Gettext macros (as opposed to functions) allows Gettext to operate on those messages *at compile-time*. This can be used to extract messages from the source code into POT (Portable Object Template) files automatically (instead of having to manually add messages to POT files when they're added to the source code). `mix gettext.extract` does exactly this: whenever there are new messages in the source code, running this task syncs the existing POT files with the changed code base. Read the documentation for `mix gettext.extract` for more information on the extraction process. POT files are just *template* files and the messages in them do not actually contain translated strings. A POT file looks like this: # The msgstr is empty msgid "hello, world" msgstr "" Whenever a POT file changes, it's likely that developers (or translators) will want to update the corresponding PO files for each locale. To do that, gettext provides the `gettext.merge` Mix task. For example, running: mix gettext.merge priv/gettext --locale pt_BR will update all the PO files in `priv/gettext/pt_BR/LC_MESSAGES` with the new version of the POT files in `priv/gettext`. Read more about the merging process in the documentation for `mix gettext.merge`. ## Configuration ### `:gettext` configuration The `:gettext` application supports the following configuration options: * `:default_locale` - a string which specifies the default global Gettext locale to use for all backends. See the "Locale" section for more information on backend-specific, global, and default locales. ### Backend configuration A **Gettext backend** supports some options to be configured. These options can be configured in two ways: either by passing them to `use Gettext` (hence at compile time): defmodule MyApp.Gettext do use Gettext.Backend, options end or by using Mix configuration, configuring the key corresponding to the backend in the configuration for your application: # For example, in config/config.exs config :my_app, MyApp.Gettext, options The `:otp_app` option (an atom representing an OTP application) has to always be present and has to be passed to `use Gettext` because it's used to determine the application to read the configuration of (`:my_app` in the example above); for this reason, `:otp_app` can't be configured via the Mix configuration. This option is also used to determine the application's directory where to search messages in. The following is a comprehensive list of supported options: * `:priv` - a string representing a directory where messages will be searched. The directory is relative to the directory of the application specified by the `:otp_app` option. It is recommended to always have this directory inside `"priv"`, otherwise some features won't work as expected. By default it's `"priv/gettext"`. * `:plural_forms` - a module which will act as a "pluralizer". For more information, look at the documentation for `Gettext.Plural`. * `:default_locale` - a string which specifies the default locale to use for the given backend. * `:split_module_by` - instead of bundling all locales into a single module, this option makes Gettext build internal modules per locale, per domain, or both. This reduces compilation times and beam file sizes for large projects. For example: `split_module_by: [:locale, :domain]`. * `:split_module_compilation` - control if compilation of split modules should happen in `:parallel` (the default) or `:serial`. * `:allowed_locales` - a list of locales to bundle in the backend. Defaults to all the locales discovered in the `:priv` directory. This option can be useful in development to reduce compile-time by compiling only a subset of all available locales. * `:interpolation` - the name of a module that implements the `Gettext.Interpolation` behaviour. Default: `Gettext.Interpolation.Default` ### Mix tasks configuration You can configure Gettext Mix tasks under the `:gettext` key in the configuration returned by `project/0` in `mix.exs`: def project() do [app: :my_app, # ... gettext: [...]] end The following is a list of the supported configuration options: * `:fuzzy_threshold` - the default threshold for the Jaro distance measuring the similarity of messages. Look at the documentation for the `mix gettext.merge` task (`Mix.Tasks.Gettext.Merge`) for more information on fuzzy messages. * `:excluded_refs_from_purging` - a regex that is matched against message references. Gettext will preserve all messages in all POT files that have a matching reference. You can use this pattern to prevent Gettext from removing messages that you have extracted using another tool. * `:custom_flags_to_keep` - a list of custom flags that will be kept for existing messages during a merge. Gettext always keeps the `fuzzy` flag. If you want to keep the `elixir-format` flag, which is also commonly used by Gettext, add it to this list. Available since v0.23.0. * `:write_reference_comments` - a boolean that specifies whether reference comments should be written when outputting PO(T) files. If this is `false`, reference comments will not be written when extracting messages or merging messages, and the ones already found in files will be discarded. * `:write_reference_line_numbers` - a boolean that specifies whether file reference comments include line numbers when outputting PO(T) files. Defaults to `true`. * `:sort_by_msgid` - modifies the sorting behavior. Can be either `nil` (the default), `:case_sensitive`, or `:case_insensitive`. By default or if `nil`, the order of existing messages in a POT file is kept and new messages are appended to the file. If `:sort_by_msgid` is set to `:case_sensitive`, existing and new messages will be mixed and sorted alphabetically by msgid. If set to `:case_insensitive`, the same applies but the sorting is case insensitive. *Note*: this option also supports `true` and `false` for backwards compatibility, but these values are deprecated as of v0.21.0. * `:on_obsolete` - controls what happens when obsolete messages are found. If `:mark_as_obsolete`, messages are kept and marked as obsolete. If `:delete`, obsolete messages are deleted. Defaults to `:delete`. * `:store_previous_message_on_fuzzy_match` - a boolean that controls whether to store the previous message text in case of a fuzzy match. Defaults to `false`. """ alias Gettext.MissingBindingsError @type locale :: binary @type backend :: module @type bindings :: map() | Keyword.t() @typedoc """ A Gettext domain. See [*Domains*](#module-domains) in the module documentation for more information. """ @typedoc since: "0.26.0" @type domain() :: :default | binary() defguardp is_domain(domain) when domain == :default or is_binary(domain) @doc false defmacro __using__(opts) do opts = if Macro.quoted_literal?(opts) do Macro.prewalk(opts, &expand_alias(&1, __CALLER__)) else opts end case Keyword.keyword?(opts) && Keyword.fetch(opts, :backend) do {:ok, backend} -> case Macro.expand(backend, __CALLER__) do backend when is_atom(backend) and backend not in [nil, false, true] -> # We need to store the module backend at expansion time because of extraction Module.put_attribute(__CALLER__.module, :__gettext_backend__, backend) quote do import Gettext.Macros end _ -> raise ArgumentError, "the :backend option on \"use Gettext\" expects the backend " <> "to be a literal atom/alias/module, got: #{Macro.to_string(backend)}" end _other -> # TODO: remove this once we stop supporting the old way of defining backends. otp_app = Keyword.get(opts, :otp_app, :my_app) IO.warn( """ Defining a Gettext backend by calling: use Gettext, otp_app: #{inspect(otp_app)} is deprecated. To define a backend, call: use Gettext.Backend, otp_app: #{inspect(otp_app)} Then, replace importing your backend: import #{inspect(__CALLER__.module)} with calling this in your module: use Gettext, backend: #{inspect(__CALLER__.module)} """, Macro.Env.stacktrace(__CALLER__) ) quote do use Gettext.Backend, unquote(opts) @before_compile {Gettext.Compiler, :generate_macros} end end end defp expand_alias({:__aliases__, _, _} = als, env) do Macro.expand(als, %{env | function: {:__gettext__, 1}}) end defp expand_alias(other, _env) do other end @doc """ Gets the global Gettext locale for the current process. This function returns the value of the global Gettext locale for the current process. This global locale is shared between all Gettext backends; if you want backend-specific locales, see `get_locale/1` and `put_locale/2`. If the global Gettext locale is not set, this function returns the default global locale (configurable in the configuration for the `:gettext` application, see the module documentation for more information). ## Examples Gettext.get_locale() #=> "en" """ @doc section: :locale @spec get_locale() :: locale def get_locale() do with nil <- Process.get(Gettext) do # If this is not set by the user, it's still set in mix.exs (to "en"). Application.fetch_env!(:gettext, :default_locale) end end @doc """ Sets the **global** Gettext locale for the current process. The locale is stored in the process dictionary. `locale` must be a string; if it's not, an `ArgumentError` exception is raised. The return value is the previous value of the current process's locale. > #### Unknown Locales {: .warning} > > Since this function sets the *global* locale, it cannot check whether that > local is supported against a particular backend. For that, use `put_locale/2` > or `put_locale!/2`. ## Examples Gettext.put_locale("pt_BR") #=> nil Gettext.get_locale() #=> "pt_BR" """ @doc section: :locale @spec put_locale(locale) :: locale | nil def put_locale(locale) when is_binary(locale), do: Process.put(Gettext, locale) def put_locale(locale), do: raise(ArgumentError, "put_locale/1 only accepts binary locales, got: #{inspect(locale)}") @doc """ Gets the locale for the current process and the given backend. This function returns the value of the locale for the current process and the given `backend`. If there is no locale for the current process and the given backend, then either the global Gettext locale (if set), or the default locale for the given backend, or the global default locale is returned. See the "Locale" section in the module documentation for more information. ## Examples Gettext.get_locale(MyApp.Gettext) #=> "en" """ @doc section: :locale @spec get_locale(backend) :: locale def get_locale(backend) do with nil <- Process.get(backend), nil <- Process.get(Gettext) do backend.__gettext__(:default_locale) end end @doc """ Sets the locale for the current process and the given `backend`. The locale is stored in the process dictionary. `locale` must be a string; if it's not, an `ArgumentError` exception is raised. The return value is the previous value of the current process's locale. ## Examples Gettext.put_locale(MyApp.Gettext, "pt_BR") #=> nil Gettext.get_locale(MyApp.Gettext) #=> "pt_BR" The current process's locale will change even if the passed `locale` is not supported. If you think this can cause an issue consider using `known_locales/1` to handle unsupported locales: # Handle unsupported locales based on your requirements defp handle_locale(locale, true, backend), do: {:ok, Process.put(backend, locale)} defp handle_locale(_locale, false, backend), do: {:error, :unsupported_locale} # In your main function is_in_allowed_locale = locale in known_locales(backend) handle_locale(locale, is_in_allowed_locale, backend) Alternatively, use `put_locale!/2` which raises if the locale is not supported. """ @doc section: :locale @spec put_locale(backend, locale) :: locale | nil def put_locale(backend, locale) when is_binary(locale), do: Process.put(backend, locale) def put_locale(_backend, locale), do: raise(ArgumentError, "put_locale/2 only accepts binary locales, got: #{inspect(locale)}") @doc """ Like `put_locale/2`, but it raises an error if the passed locale doesn't exist in the known locales. ## Examples Gettext.put_locale(MyApp.Gettext, "pt_BR") #=> nil Gettext.get_locale(MyApp.Gettext) #=> "pt_BR" """ @doc section: :locale @doc since: "1.0.0" @spec put_locale!(backend, locale) :: locale | nil def put_locale!(backend, locale) when is_binary(locale) do cond do not is_binary(locale) -> raise ArgumentError, "put_locale/2 only accepts binary locales, got: #{inspect(locale)}" locale in known_locales(backend) -> put_locale(backend, locale) true -> raise ArgumentError, "put_locale!/2 only support known locales, got: #{inspect(locale)}" end end @doc """ Returns the message of the given string with a given context in the given domain. The string is translated by the `backend` module. The translated string is interpolated based on the `bindings` argument. For more information on how interpolation works, refer to the documentation of the `Gettext` module. If the message for the given `msgid` is not found, the `msgid` (interpolated if necessary) is returned. ## Examples defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Gettext.put_locale(MyApp.Gettext, "it") Gettext.dpgettext(MyApp.Gettext, "errors", "user error", "Invalid") #=> "Non valido" Gettext.dgettext(MyApp.Gettext, "errors", "signup form", "%{name} is not a valid name", name: "Meg") #=> "Meg non è un nome valido" """ @doc section: :translation @spec dpgettext(module, domain, binary | nil, binary, bindings) :: binary def dpgettext(backend, domain, msgctxt, msgid, bindings \\ %{}) def dpgettext(backend, domain, msgctxt, msgid, bindings) when is_list(bindings) do dpgettext(backend, domain, msgctxt, msgid, Map.new(bindings)) end def dpgettext(backend, domain, msgctxt, msgid, bindings) when is_atom(backend) and is_domain(domain) and is_binary(msgid) and is_map(bindings) do domain = domain_or_default(backend, domain) locale = get_locale(backend) result = backend.lgettext(locale, domain, msgctxt, msgid, bindings) handle_backend_result(result, backend, locale, domain, msgctxt, msgid) end @doc """ Returns the message of the given string in the given domain. The string is translated by the `backend` module. The translated string is interpolated based on the `bindings` argument. For more information on how interpolation works, refer to the documentation of the `Gettext` module. If the message for the given `msgid` is not found, the `msgid` (interpolated if necessary) is returned. ## Examples defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Gettext.put_locale(MyApp.Gettext, "it") Gettext.dgettext(MyApp.Gettext, "errors", "Invalid") #=> "Non valido" Gettext.dgettext(MyApp.Gettext, "errors", "%{name} is not a valid name", name: "Meg") #=> "Meg non è un nome valido" Gettext.dgettext(MyApp.Gettext, "alerts", "nonexisting") #=> "nonexisting" """ @doc section: :translation @spec dgettext(module, domain, binary, bindings) :: binary def dgettext(backend, domain, msgid, bindings \\ %{}) do dpgettext(backend, domain, nil, msgid, bindings) end @doc """ Returns the message of the given string with the given context The string is translated by the `backend` module. The translated string is interpolated based on the `bindings` argument. For more information on how interpolation works, refer to the documentation of the `Gettext` module. If the message for the given `msgid` is not found, the `msgid` (interpolated if necessary) is returned. ## Examples defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Gettext.put_locale(MyApp.Gettext, "it") Gettext.pgettext(MyApp.Gettext, "user-interface", "Invalid") #=> "Non valido" Gettext.pgettext(MyApp.Gettext, "user-interface", "%{name} is not a valid name", name: "Meg") #=> "Meg non è un nome valido" Gettext.pgettext(MyApp.Gettext, "alerts-users", "nonexisting") #=> "nonexisting" """ @doc section: :translation @spec pgettext(module, binary, binary, bindings) :: binary def pgettext(backend, msgctxt, msgid, bindings \\ %{}) do dpgettext(backend, "default", msgctxt, msgid, bindings) end @doc """ Returns the message of the given string in the `"default"` domain. Works exactly like: Gettext.dgettext(backend, "default", msgid, bindings) """ @doc section: :translation @spec gettext(module, binary, bindings) :: binary def gettext(backend, msgid, bindings \\ %{}) do dgettext(backend, "default", msgid, bindings) end @doc """ Returns the pluralized message of the given string with a given context in the given domain. The string is translated and pluralized by the `backend` module. The translated string is interpolated based on the `bindings` argument. For more information on how interpolation works, refer to the documentation of the `Gettext` module. If the message for the given `msgid` and `msgid_plural` is not found, the `msgid` or `msgid_plural` (based on `n` being singular or plural) is returned (interpolated if necessary). ## Examples defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Gettext.dpngettext(MyApp.Gettext, "errors", "user error", "Error", "%{count} errors", 3) #=> "3 errori" Gettext.dpngettext(MyApp.Gettext, "errors", "user error", "Error", "%{count} errors", 1) #=> "Errore" """ @doc section: :translation @spec dpngettext(module, domain, binary | nil, binary, binary, non_neg_integer, bindings) :: binary def dpngettext(backend, domain, msgctxt, msgid, msgid_plural, n, bindings \\ %{}) def dpngettext(backend, domain, msgctxt, msgid, msgid_plural, n, bindings) when is_list(bindings) do dpngettext(backend, domain, msgctxt, msgid, msgid_plural, n, Map.new(bindings)) end def dpngettext(backend, domain, msgctxt, msgid, msgid_plural, n, bindings) when is_atom(backend) and is_domain(domain) and is_binary(msgid) and is_binary(msgid_plural) and is_integer(n) and n >= 0 and is_map(bindings) do domain = domain_or_default(backend, domain) locale = get_locale(backend) result = backend.lngettext(locale, domain, msgctxt, msgid, msgid_plural, n, bindings) handle_backend_result(result, backend, locale, domain, msgctxt, msgid) end @doc """ Returns the pluralized message of the given string in the given domain. The string is translated and pluralized by the `backend` module. The translated string is interpolated based on the `bindings` argument. For more information on how interpolation works, refer to the documentation of the `Gettext` module. If the message for the given `msgid` and `msgid_plural` is not found, the `msgid` or `msgid_plural` (based on `n` being singular or plural) is returned (interpolated if necessary). ## Examples defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end Gettext.dngettext(MyApp.Gettext, "errors", "Error", "%{count} errors", 3) #=> "3 errori" Gettext.dngettext(MyApp.Gettext, "errors", "Error", "%{count} errors", 1) #=> "Errore" """ @doc section: :translation @spec dngettext(module, domain, binary, binary, non_neg_integer, bindings) :: binary def dngettext(backend, domain, msgid, msgid_plural, n, bindings \\ %{}), do: dpngettext(backend, domain, nil, msgid, msgid_plural, n, bindings) @doc """ Returns the pluralized message of the given string with a given context in the `"default"` domain. Works exactly like: Gettext.dpngettext(backend, "default", context, msgid, msgid_plural, n, bindings) """ @doc section: :translation @spec pngettext(module, binary, binary, binary, non_neg_integer, bindings) :: binary def pngettext(backend, msgctxt, msgid, msgid_plural, n, bindings), do: dpngettext(backend, "default", msgctxt, msgid, msgid_plural, n, bindings) @doc """ Returns the pluralized message of the given string in the `"default"` domain. Works exactly like: Gettext.dngettext(backend, "default", msgid, msgid_plural, n, bindings) """ @doc section: :translation @spec ngettext(module, binary, binary, non_neg_integer, bindings) :: binary def ngettext(backend, msgid, msgid_plural, n, bindings \\ %{}) do dngettext(backend, "default", msgid, msgid_plural, n, bindings) end @doc """ Runs `fun` with the global Gettext locale set to `locale`. This function just sets the global Gettext locale to `locale` before running `fun` and sets it back to its previous value afterwards. Note that `put_locale/2` is used to set the locale, which is thus set only for the current process (keep this in mind if you plan on spawning processes inside `fun`). The value returned by this function is the return value of `fun`. ## Examples Gettext.put_locale("fr") gettext("Hello world") #=> "Bonjour monde" Gettext.with_locale("it", fn -> gettext("Hello world") end) #=> "Ciao mondo" gettext("Hello world") #=> "Bonjour monde" """ @doc section: :locale @spec with_locale(locale, (-> result)) :: result when result: var def with_locale(locale, fun) when is_binary(locale) and is_function(fun) do previous_locale = Process.get(Gettext) Gettext.put_locale(locale) try do fun.() after if previous_locale do Gettext.put_locale(previous_locale) else Process.delete(Gettext) end end end @doc """ Runs `fun` with the Gettext locale set to `locale` for the given `backend`. This function just sets the Gettext locale for `backend` to `locale` before running `fun` and sets it back to its previous value afterwards. Note that `put_locale/2` is used to set the locale, which is thus set only for the current process (keep this in mind if you plan on spawning processes inside `fun`). The value returned by this function is the return value of `fun`. ## Examples Gettext.put_locale(MyApp.Gettext, "fr") gettext("Hello world") #=> "Bonjour monde" Gettext.with_locale(MyApp.Gettext, "it", fn -> gettext("Hello world") end) #=> "Ciao mondo" gettext("Hello world") #=> "Bonjour monde" """ @doc section: :locale @spec with_locale(backend(), locale(), (-> result)) :: result when result: var def with_locale(backend, locale, fun) when is_atom(backend) and is_binary(locale) and is_function(fun) do previous_locale = Process.get(backend) Gettext.put_locale(backend, locale) try do fun.() after if previous_locale do Gettext.put_locale(backend, previous_locale) else Process.delete(backend) end end end @doc """ Returns all the locales for which PO files exist for the given `backend`. If the messages directory for the given backend doesn't exist, then an empty list is returned. ## Examples With the following backend: defmodule MyApp.Gettext do use Gettext.Backend, otp_app: :my_app end and the following messages directory: my_app/priv/gettext ├─ en ├─ it └─ pt_BR then: Gettext.known_locales(MyApp.Gettext) #=> ["en", "it", "pt_BR"] """ @doc section: :locale @spec known_locales(backend()) :: [locale()] def known_locales(backend) when is_atom(backend) do backend.__gettext__(:known_locales) end defp handle_backend_result({:ok, string}, _backend, _locale, _domain, _msgctxt, _msgid) do string end defp handle_backend_result({:default, string}, _backend, _locale, _domain, _msgctxt, _msgid) do string end defp handle_backend_result( {:missing_bindings, incomplete, missing}, backend, locale, domain, msgctxt, msgid ) do exception = %MissingBindingsError{ backend: backend, locale: locale, domain: domain, msgctxt: msgctxt, msgid: msgid, missing: missing } backend.handle_missing_bindings(exception, incomplete) end defp domain_or_default(backend, :default), do: backend.__gettext__(:default_domain) defp domain_or_default(_backend, domain) when is_binary(domain), do: domain end ================================================ FILE: lib/mix/tasks/compile.gettext.ex ================================================ defmodule Mix.Tasks.Compile.Gettext do @moduledoc false def run(_args) do IO.warn(""" the :gettext compiler is no longer required in your mix.exs. Please find the following line in your mix.exs and remove the :gettext entry: compilers: [..., :gettext, ...] ++ Mix.compilers(), """) {:noop, []} end end ================================================ FILE: lib/mix/tasks/gettext.extract.ex ================================================ defmodule Mix.Tasks.Gettext.Extract do use Mix.Task @recursive true @shortdoc "Extracts messages from source code" @moduledoc """ Extracts messages by recompiling the Elixir source code. ```bash mix gettext.extract [OPTIONS] ``` messages are extracted into POT (Portable Object Template) files with a `.pot` extension. The location of these files is determined by the `:otp_app` and `:priv` options given by Gettext modules when they call `use Gettext`. One POT file is generated for each message domain. All automatically-extracted messages are assigned the `elixir-autogen` flag. If a message from the POT is no longer present and has the `elixir-autogen` flag, the message is removed. Before `v0.19.0`, the `elixir-format` flag was used to detect automatically extracted messages. This has been deprecated in `v0.19.0`. When extracting with the newest version, the new `elixir-autogen` flag is added to all automatically extracted messages. All messages are assigned a format flag. When using the default interpolation module, that flag is `elixir-format`. With other interpolation modules, the flag name is defined by that implementation (see `c:Gettext.Interpolation.message_format/0`). If you would like to verify that your POT files are up to date with the current state of the codebase, you can provide the `--check-up-to-date` flag. This is particularly useful for automated checks and in CI systems. This validation will fail even when the same calls to Gettext only change location in the codebase: ```bash mix gettext.extract --check-up-to-date ``` It is possible to pass the `--merge` option to perform merging for every Gettext backend updated during merge: ```bash mix gettext.extract --merge ``` All other options passed to `gettext.extract` are forwarded to the `gettext.merge` task (`Mix.Tasks.Gettext.Merge`), which is called internally by this task. For example: ```bash mix gettext.extract --merge --no-fuzzy ``` """ @switches [merge: :boolean, check_up_to_date: :boolean] @impl true def run(args) do Application.ensure_all_started(:gettext) _ = Mix.Project.get!() mix_config = Mix.Project.config() {opts, _} = OptionParser.parse!(args, switches: @switches) pot_files = extract(mix_config[:app], mix_config[:gettext] || []) if opts[:check_up_to_date] do run_up_to_date_check(pot_files) else run_message_extraction(pot_files, opts, args) end end defp run_message_extraction(pot_files, opts, args) do for {path, {:changed, contents}} <- pot_files do File.mkdir_p!(Path.dirname(path)) File.write!(path, contents) Mix.shell().info("Extracted #{Path.relative_to_cwd(path)}") end if opts[:merge] do run_merge(pot_files, args) end :ok end defp run_up_to_date_check(pot_files) do not_extracted_paths = for {path, {:changed, _contents}} <- pot_files, do: path if not_extracted_paths == [] do :ok else Mix.raise(""" mix gettext.extract failed due to --check-up-to-date. The following POT files were not extracted or are out of date: #{Enum.map_join(not_extracted_paths, "\n", &" * #{&1 |> Path.relative_to_cwd()}")} """) end end defp extract(app, gettext_config) do Gettext.Extractor.enable() force_compile() Gettext.Extractor.pot_files(app, gettext_config) after Gettext.Extractor.disable() end defp force_compile do # For old Elixir versions, we have to clean the manifest, # otherwise we are forced to compile all dependencies. # Elixir v1.19.3 supports the --force-elixir option below. if not Version.match?(System.version(), ">= 1.19.3") do Mix.Tasks.Compile.Elixir.clean() Enum.each(Mix.Tasks.Compile.Elixir.manifests(), &File.rm/1) end # If "compile" was never called, the reenabling is a no-op and # "compile.elixir" is a no-op as well (because it wasn't reenabled after # running "compile"). If "compile" was already called, then running # "compile" is a no-op and running "compile.elixir" will work because we # manually reenabled it. Mix.Task.reenable("compile.elixir") Mix.Task.run("compile", ["--force-elixir"]) Mix.Task.run("compile.elixir", ["--force"]) end defp run_merge(pot_files, argv) do pot_files |> Enum.map(fn {path, _} -> Path.dirname(path) end) |> Enum.uniq() |> Task.async_stream(&Mix.Tasks.Gettext.Merge.run([&1 | argv]), ordered: false, timeout: 120_000 ) |> Stream.run() end end ================================================ FILE: lib/mix/tasks/gettext.merge.ex ================================================ defmodule Mix.Tasks.Gettext.Merge do use Mix.Task @recursive true @shortdoc "Merge template files into message files" @moduledoc """ Merges PO/POT files with PO files. This task is used when messages in the source code change: when they do, `mix gettext.extract` is usually used to extract the new messages to POT files. At this point, developers or translators can use this task to "sync" the newly-updated POT files with the existing locale-specific PO files. All the metadata for each message (like position in the source code, comments, and so on) is taken from the newly-updated POT file; the only things taken from the PO file are the actual translated strings. #### Fuzzy Matching Messages in the updated PO/POT file that have an exact match (a message with the same `msgid`) in the old PO file are merged as described above. When a message in the updated PO/POT files has no match in the old PO file, Gettext attemps a **fuzzy match** for that message. For example, imagine we have this POT file: msgid "hello, world!" msgstr "" and we merge it with this PO file: # No exclamation point here in the msgid msgid "hello, world" msgstr "ciao, mondo" Since the two messages are similar, Gettext takes the `msgstr` from the existing message over to the new message, which it however marks as *fuzzy*: #, fuzzy msgid "hello, world!" msgstr "ciao, mondo" Generally, a `fuzzy` flag calls for review from a translator. Fuzzy matching can be configured (for example, the threshold for message similarity can be tweaked) or disabled entirely. Look at the ["Options" section](#module-options). ## Usage ```bash mix gettext.merge OLD_FILE UPDATED_FILE [OPTIONS] mix gettext.merge DIR [OPTIONS] ``` If two files are given as arguments, `OLD_FILE` must be a `.po` file and `UPDATE_FILE` must be a `.po`/`.pot` file. The first one is the old PO file, while the second one is the last generated one. They are merged and written over the first file. For example: ```bash mix gettext.merge priv/gettext/en/LC_MESSAGES/default.po priv/gettext/default.pot ``` If only one argument is given, then that argument must be a directory containing Gettext messages (with `.pot` files at the root level alongside locale directories - this is usually a "backend" directory used by a Gettext backend, see `Gettext.Backend`). For example: ```bash mix gettext.merge priv/gettext ``` If the `--locale LOCALE` option is given, then only the PO files in `//LC_MESSAGES` will be merged with the POT files in `DIR`. If no options are given, then all the PO files for all locales under `DIR` are merged with the POT files in `DIR`. ## Plural Forms By default, Gettext will determine the number of plural forms for newly-generated messages by checking the value of `nplurals` in the `Plural-Forms` header in the existing `.po` file. If a `.po` file doesn't already exist and Gettext is creating a new one or if the `Plural-Forms` header is not in the `.po` file, Gettext will use the number of plural forms that the plural module (see `Gettext.Plural`) returns for the locale of the file being created. The content of the `Plural-Forms` header can be forced through the `--plural-forms-header` option (see below). ## Options * `--locale` - a string representing a locale. If this is provided, then only the PO files in `//LC_MESSAGES` will be merged with the POT files in `DIR`. This option can only be given when a single argument is passed to the task (a directory). * `--no-fuzzy` - don't perform fuzzy matching when merging files. * `--fuzzy-threshold` - a float between `0` and `1` which represents the minimum Jaro distance needed for two messages to be considered a fuzzy match. Overrides the global `:fuzzy_threshold` option (see the docs for `Gettext` for more information on this option). * `--plural-forms` - (**deprecated in v0.22.0**) an integer strictly greater than `0`. If this is passed, new messages in the target PO files will have this number of empty plural forms. This is deprecated in favor of passing the `--plural-forms-header`, which contains the whole plural-forms specification. See the "Plural forms" section above. * `--plural-forms-header` - the content of the `Plural-Forms` header as a string. If this is passed, new messages in the target PO files will use this content to determine the number of plurals. See the ["Plural Forms" section](#module-plural-forms). * `--on-obsolete` - controls what happens when **obsolete** messages are found. If `mark_as_obsolete`, messages are kept and marked as obsolete. If `delete`, obsolete messages are deleted. Defaults to `delete`. * `--store-previous-message-on-fuzzy-match` - controls if the previous messages are recorded on fuzzy matches. Is off by default. """ alias Expo.PO alias Gettext.Merger @default_fuzzy_threshold 0.8 @switches [ locale: :string, fuzzy: :boolean, fuzzy_threshold: :float, plural_forms_header: :string, on_obsolete: :string, store_previous_message_on_fuzzy_match: :boolean ] @impl true def run(args) do Mix.Task.run("loadpaths") _ = Mix.Project.get!() gettext_config = Mix.Project.config()[:gettext] || [] case OptionParser.parse!(args, switches: @switches) do {opts, [po_file, reference_file]} -> merge_two_files(po_file, reference_file, opts, gettext_config) {opts, [messages_dir]} -> merge_messages_dir(messages_dir, opts, gettext_config) {_opts, _args} -> Mix.raise( "You can only pass one or two arguments to the \"gettext.merge\" task. " <> "Use `mix help gettext.merge` to see the usage of this task" ) end Mix.Task.reenable("gettext.merge") end defp merge_two_files(po_file, reference_file, opts, gettext_config) do merging_opts = validate_merging_opts!(opts, gettext_config) if Path.extname(po_file) == ".po" and Path.extname(reference_file) in [".po", ".pot"] do ensure_file_exists!(po_file) ensure_file_exists!(reference_file) locale = locale_from_path(po_file) {contents, stats} = merge_files(po_file, reference_file, locale, merging_opts, gettext_config) write_file(po_file, contents, stats) else Mix.raise("Arguments must be a PO file and a PO/POT file") end end defp merge_messages_dir(dir, opts, gettext_config) do ensure_dir_exists!(dir) merging_opts = validate_merging_opts!(opts, gettext_config) if locale = opts[:locale] do merge_locale_dir(dir, locale, merging_opts, gettext_config) else merge_all_locale_dirs(dir, merging_opts, gettext_config) end end defp merge_locale_dir(pot_dir, locale, opts, gettext_config) do locale_dir = locale_dir(pot_dir, locale) create_missing_locale_dir(locale_dir) merge_dirs(locale_dir, pot_dir, locale, opts, gettext_config) end defp merge_all_locale_dirs(pot_dir, opts, gettext_config) do for locale <- File.ls!(pot_dir), File.dir?(Path.join(pot_dir, locale)) do merge_dirs(locale_dir(pot_dir, locale), pot_dir, locale, opts, gettext_config) end end def locale_dir(pot_dir, locale) do Path.join([pot_dir, locale, "LC_MESSAGES"]) end defp merge_dirs(po_dir, pot_dir, locale, opts, gettext_config) do merger = fn pot_file -> po_file = find_matching_po(pot_file, po_dir) {contents, stats} = merge_or_create(pot_file, po_file, locale, opts, gettext_config) write_file(po_file, contents, stats) end pot_dir |> Path.join("*.pot") |> Path.wildcard() |> Task.async_stream(merger, ordered: false, timeout: :infinity) |> Stream.run() warn_for_po_without_pot(po_dir, pot_dir) end defp find_matching_po(pot_file, po_dir) do domain = Path.basename(pot_file, ".pot") Path.join(po_dir, "#{domain}.po") end defp merge_or_create(pot_file, po_file, locale, opts, gettext_config) do if File.regular?(po_file) do merge_files(po_file, pot_file, locale, opts, gettext_config) else {new_po, stats} = Merger.new_po_file(po_file, pot_file, locale, opts) {new_po |> Merger.prune_references(gettext_config) |> PO.compose(), stats} end end defp merge_files(po_file, pot_file, locale, opts, gettext_config) do {merged, stats} = Merger.merge( PO.parse_file!(po_file), PO.parse_file!(pot_file), locale, opts, gettext_config ) {merged |> Merger.prune_references(gettext_config) |> PO.compose(), stats} end defp write_file(path, contents, stats) do File.mkdir_p!(Path.dirname(path)) File.write!(path, contents) Mix.shell().info("Wrote #{path} (#{format_stats(stats)})") end # Warns for every PO file that has no matching POT file. defp warn_for_po_without_pot(po_dir, pot_dir) do po_dir |> Path.join("*.po") |> Path.wildcard() |> Enum.reject(&po_has_matching_pot?(&1, pot_dir)) |> Enum.each(fn po_file -> Mix.shell().info("Warning: PO file #{po_file} has no matching POT file in #{pot_dir}") end) end defp po_has_matching_pot?(po_file, pot_dir) do domain = Path.basename(po_file, ".po") pot_path = Path.join(pot_dir, "#{domain}.pot") File.exists?(pot_path) end defp ensure_file_exists!(path) do unless File.regular?(path), do: Mix.raise("No such file: #{path}") end defp ensure_dir_exists!(path) do unless File.dir?(path), do: Mix.raise("No such directory: #{path}") end defp create_missing_locale_dir(dir) do unless File.dir?(dir) do File.mkdir_p!(dir) Mix.shell().info("Created directory #{dir}") end end defp validate_merging_opts!(opts, gettext_config) do opts = opts |> Keyword.take([ :fuzzy, :fuzzy_threshold, :plural_forms_header, :on_obsolete, :store_previous_message_on_fuzzy_match ]) |> Keyword.put_new(:store_previous_message_on_fuzzy_match, false) |> Keyword.put_new(:fuzzy, true) |> Keyword.put_new_lazy(:fuzzy_threshold, fn -> gettext_config[:fuzzy_threshold] || @default_fuzzy_threshold end) |> Keyword.update(:on_obsolete, :delete, &cast_on_obsolete/1) threshold = opts[:fuzzy_threshold] unless threshold >= 0.0 and threshold <= 1.0 do Mix.raise("The :fuzzy_threshold option must be a float >= 0.0 and <= 1.0") end opts end defp locale_from_path(path) do parts = Path.split(path) index = Enum.find_index(parts, &(&1 == "LC_MESSAGES")) Enum.at(parts, index - 1) end defp format_stats(stats) do pluralized = if stats.new == 1, do: "message", else: "messages" "#{stats.new} new #{pluralized}, #{stats.removed} removed, " <> "#{stats.exact_matches} unchanged, #{stats.fuzzy_matches} reworded (fuzzy), " <> "#{stats.marked_as_obsolete} marked as obsolete" end defp cast_on_obsolete("delete" = _on_obsolete), do: :delete defp cast_on_obsolete("mark_as_obsolete" = _on_obsolete), do: :mark_as_obsolete defp cast_on_obsolete(on_obsolete) do Mix.raise(""" An invalid value was provided for the option `on_obsolete`. Value: #{inspect(on_obsolete)} Valid Choices: "delete" / "mark_as_obsolete" """) end end ================================================ FILE: mix.exs ================================================ defmodule Gettext.Mixfile do use Mix.Project @version "1.0.2" @description "Internationalization and localization through gettext" @repo_url "https://github.com/elixir-gettext/gettext" def project do [ app: :gettext, version: @version, elixir: "~> 1.14", elixirc_paths: elixirc_paths(Mix.env()), build_embedded: Mix.env() == :prod, deps: deps(), preferred_cli_env: [coveralls: :test, "coveralls.html": :test, "coveralls.github": :test], test_coverage: [tool: ExCoveralls], # Hex package: hex_package(), description: @description, # Docs name: "gettext", docs: [ source_ref: "v#{@version}", main: "Gettext", source_url: @repo_url, extras: ["CHANGELOG.md"], groups_for_docs: [ # Gettext "Translation Functions": &(&1[:section] == :translation), "Locale Functions": &(&1[:section] == :locale), # Gettext.Macros "Macros with Backend": &(&1[:module] == Gettext.Macros and to_string(&1[:name]) =~ ~r/_with_backend$/), "Comment Macros": &(&1[:module] == Gettext.Macros and &1[:name] == :gettext_comment), "Extraction Macros": &(&1[:module] == Gettext.Macros and to_string(&1[:name]) =~ ~r/_noop$/), "Translation Macros": &(&1[:module] == Gettext.Macros) ] ] ] end def application do [ extra_applications: [:logger], env: [default_locale: "en", plural_forms: Gettext.Plural], mod: {Gettext.Application, []} ] end def hex_package do [ maintainers: ["Andrea Leopardi", "Jonatan Männchen", "José Valim"], licenses: ["Apache-2.0"], links: %{ "GitHub" => @repo_url, "Changelog" => @repo_url <> "/blob/main/CHANGELOG.md" }, files: ~w(lib mix.exs *.md) ] end defp elixirc_paths(:test), do: ["lib", "test/support"] defp elixirc_paths(_other), do: ["lib"] defp deps do [ {:expo, "~> 0.5.1 or ~> 1.0"}, # Dev and test dependencies {:castore, "~> 1.0", only: :test}, {:jason, "~> 1.0", only: :test}, {:ex_doc, "~> 0.19", only: :dev}, {:excoveralls, "~> 0.18.0", only: :test} ] end end ================================================ FILE: test/fixtures/bad_messages/ru/LC_MESSAGES/errors.po ================================================ # Russian has 3 plural forms, only 2 are defined here. msgid "should be at least %{count} character(s)" msgid_plural "should be at least %{count} character(s)" msgstr[0] "Требуется минимум один символ" msgstr[1] "Требуется минимум %{count} символов" ================================================ FILE: test/fixtures/bom.po ================================================ # NOTE: the BOM sequence isn't visible in most editors. Just don't change this # file :) msgid "foo" msgstr "bar" ================================================ FILE: test/fixtures/empty.po ================================================ ================================================ FILE: test/fixtures/invalid_syntax_error.po ================================================ msgid "foo" msgstr "bar" msgstr "bong" ================================================ FILE: test/fixtures/invalid_token_error.po ================================================ msg ================================================ FILE: test/fixtures/multi_messages/es/LC_MESSAGES/default.po ================================================ msgid "Hello world" msgstr "Hola mundo" ================================================ FILE: test/fixtures/multi_messages/it/LC_MESSAGES/default.po ================================================ msgid "Hello world" msgstr "Ciao mondo" msgctxt "test" msgid "Hello world" msgstr "Ciao mondo" ================================================ FILE: test/fixtures/multi_messages/it/LC_MESSAGES/errors.po ================================================ msgid "Invalid email address" msgstr "Indirizzo email non valido" ================================================ FILE: test/fixtures/po_editors/poedit.po ================================================ msgid "" msgstr "" "Project-Id-Version: POEdit test project\n" "POT-Creation-Date: \n" "PO-Revision-Date: \n" "Last-Translator: Meg Buque\n" "Language-Team: Elixir \n" "Language: it\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" "X-Generator: Poedit 1.7.6\n" "Plural-Forms: nplurals=2; plural=(n != 1);\n" "X-Poedit-SourceCharset: UTF-8\n" msgid "Hello" msgstr "Ciao" msgid "Hello, %{name}!" msgstr "Ciao, %{name}!" msgid "One error" msgid_plural "%{count} errors" msgstr[0] "Un errore" msgstr[1] "%{count} errori" msgid "String with \"double quotes\" in it" msgstr "Stringa che contiene \"doppie virgolette\"" ================================================ FILE: test/fixtures/po_editors/poeditor.com.po ================================================ msgid "" msgstr "" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" "X-Generator: POEditor.com\n" "Project-Id-Version: POEditor testing\n" "Language: it\n" "Plural-Forms: nplurals=2; plural=(n != 1);\n" #. A comment msgid "Hello" msgstr "Ciao" msgid "Hello, %{name}!" msgstr "Ciao, %{name}!" #. A plural message msgid "One error" msgid_plural "%{count} errors" msgstr[0] "Un errore" msgstr[1] "%{count} errori" msgid "String with \"double quotes\" in it" msgstr "Stringa che contiene \"doppie virgolette\"" ================================================ FILE: test/fixtures/single_messages/it/LC_MESSAGES/default.po ================================================ msgid "" msgstr "" "Language: it\n" "Plural-Forms: nplurals=2; plural=(n != 1);\n" msgid "Hello world" msgstr "Ciao mondo" msgctxt "test" msgid "Hello world" msgstr "Ciao mondo" msgctxt "test" msgid "Hello %{name}" msgstr "Ciao %{name}" msgid "One new email" msgid_plural "%{count} new emails" msgstr[0] "Una nuova email" msgstr[1] "%{count} nuove email" msgctxt "test" msgid "One new email" msgid_plural "%{count} new emails" msgstr[0] "Una nuova test email" msgstr[1] "%{count} nuove test email" msgid "Concatenated" " and long " "string" msgstr "Stringa" " lunga e " "concatenata" msgid "A" " friend" msgid_plural "%{count}" " friends" msgstr[0] "Un" " amico" msgstr[1] "%{count}" " amici" msgid "Empty msgstr!" msgstr "" "" msgid "Not even one msgstr" msgid_plural "Not even %{count} msgstrs" msgstr[0] "" msgstr[1] "" ================================================ FILE: test/fixtures/single_messages/it/LC_MESSAGES/errors.po ================================================ msgid "Invalid email address" msgstr "Indirizzo email non valido" msgid "There was an error" msgid_plural "There were %{count} errors" msgstr[0] "C'è stato un errore" msgstr[1] "Ci sono stati %{count} errori" ================================================ FILE: test/fixtures/single_messages/it/LC_MESSAGES/interpolations.po ================================================ msgid "Hello %{name}" msgstr "Ciao %{name}" msgid "My name is %{name} and I'm %{age}" msgstr "Mi chiamo %{name} e ho %{age} anni" msgid "You have one message, %{name}" msgid_plural "You have %{count} messages, %{name}" msgstr[0] "Hai un messaggio, %{name}" msgstr[1] "Hai %{count} messaggi, %{name}" msgid "Month" msgid_plural "%{count} months" msgstr[0] "Mese" msgstr[1] "%{count} mesi" msgctxt "test" msgid "You have one message, %{name}" msgid_plural "You have %{count} messages, %{name}" msgstr[0] "Hai un messaggio, %{name}" msgstr[1] "Hai %{count} messaggi, %{name}" ================================================ FILE: test/fixtures/single_messages/ja/LC_MESSAGES/errors.po ================================================ msgid "Invalid email address" msgstr "無効なメールアドレス" msgid "There was an error" msgid_plural "There were %{count} errors" msgstr[0] "%{count} エラーがありました" ================================================ FILE: test/fixtures/valid.po ================================================ msgid "hello" msgstr "ciao" msgid "how are you," " friend?" msgstr "come stai," " amico?" ================================================ FILE: test/gettext/backend_test.exs ================================================ defmodule Gettext.BackendTest do # Some things change the :gettext app environment. use ExUnit.Case, async: false import ExUnit.CaptureLog alias GettextTest.{ Backend, BackendWithDefaultDomain } defmodule BackendWithCustomPluralForms do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/single_messages", plural_forms: GettextTest.CustomPlural end defmodule BackendWithCustomCompiledPluralForms do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/single_messages", plural_forms: GettextTest.CustomCompiledPlural end defmodule BackendWithOneModulePerLocale do use Gettext.Backend, otp_app: :test_application, split_module_by: [:locale], split_module_compilation: :parallel, priv: "test/fixtures/single_messages" end defmodule BackendWithOneModulePerLocaleDomain do use Gettext.Backend, otp_app: :test_application, split_module_by: [:locale, :domain], split_module_compilation: :serial, priv: "test/fixtures/single_messages" end describe "use Gettext.Backend" do test "creates a backend" do body = quote do use Gettext.Backend, otp_app: :test_application end {:module, mod, _bytecode, :ok} = Module.create(TestBackend, body, __ENV__) assert mod.__gettext__(:otp_app) == :test_application assert mod.__info__(:attributes)[:behaviour] == [Gettext.Backend] end test "may define one module per locale" do import BackendWithOneModulePerLocale, only: [lgettext: 5, lngettext: 7] assert Code.ensure_loaded?(BackendWithOneModulePerLocale.T_it) # Found on default domain. assert lgettext("it", "default", nil, "Hello world", %{}) == {:ok, "Ciao mondo"} # Found on errors domain. assert lgettext("it", "errors", nil, "Invalid email address", %{}) == {:ok, "Indirizzo email non valido"} # Found with plural form. assert lngettext( "it", "errors", nil, "There was an error", "There were %{count} errors", 1, %{} ) == {:ok, "C'è stato un errore"} # Unknown msgid. assert lgettext("it", "default", nil, "nonexistent", %{}) == {:default, "nonexistent"} # Unknown domain. assert lgettext("it", "unknown", nil, "Hello world", %{}) == {:default, "Hello world"} # Unknown locale. assert lgettext("pt_BR", "nonexistent", nil, "Hello world", %{}) == {:default, "Hello world"} end test "may define one module per locale and domain" do import BackendWithOneModulePerLocaleDomain, only: [lgettext: 5, lngettext: 7] assert Code.ensure_loaded?(BackendWithOneModulePerLocaleDomain.T_it_default) # Found on default domain. assert lgettext("it", "default", nil, "Hello world", %{}) == {:ok, "Ciao mondo"} # Found on errors domain. assert lgettext("it", "errors", nil, "Invalid email address", %{}) == {:ok, "Indirizzo email non valido"} # Found with plural form. assert lngettext( "it", "errors", nil, "There was an error", "There were %{count} errors", 1, %{} ) == {:ok, "C'è stato un errore"} # Unknown msgid. assert lgettext("it", "default", nil, "nonexistent", %{}) == {:default, "nonexistent"} # Unknown domain. assert lgettext("it", "unknown", nil, "Hello world", %{}) == {:default, "Hello world"} # Unknown locale. assert lgettext("pt_BR", "nonexistent", nil, "Hello world", %{}) == {:default, "Hello world"} end end describe "__gettext__/1 (generated)" do test "with :priv returns the directory where messages are stored" do assert Backend.__gettext__(:priv) == "test/fixtures/single_messages" end test "with :otp_app returns the OTP app for the given backend" do assert Backend.__gettext__(:otp_app) == :test_application end test "with :default_domain returns the default domain for the given backend" do assert Backend.__gettext__(:default_domain) == "default" assert BackendWithDefaultDomain.__gettext__(:default_domain) == "errors" end end describe "c:lgettext/5" do test "returns {:ok, translation} for found translations" do assert Backend.lgettext("it", "default", nil, "Hello world", %{}) == {:ok, "Ciao mondo"} assert Backend.lgettext("it", "errors", nil, "Invalid email address", %{}) == {:ok, "Indirizzo email non valido"} end test "returns {:default, msgid} for missing translations" do # Unknown msgid. assert Backend.lgettext("it", "default", nil, "nonexistent", %{}) == {:default, "nonexistent"} # Unknown domain. assert Backend.lgettext("it", "unknown", nil, "Hello world", %{}) == {:default, "Hello world"} # Unknown locale. assert Backend.lgettext("pt_BR", "nonexistent", nil, "Hello world", %{}) == {:default, "Hello world"} end test "returns {:default, msgid} if the msgstr is an empty string" do assert Backend.lgettext("it", "default", nil, "Empty msgstr!", %{}) == {:default, "Empty msgstr!"} end test "supports translating the msgid of plural translations" do assert Backend.lgettext("it", "errors", nil, "There was an error", %{}) == {:ok, "C'è stato un errore"} end test "supports interpolation with found translations" do assert Backend.lgettext("it", "interpolations", nil, "Hello %{name}", %{name: "Jane"}) == {:ok, "Ciao Jane"} msgid = "My name is %{name} and I'm %{age}" assert Backend.lgettext("it", "interpolations", nil, msgid, %{name: "Meg", age: 33}) == {:ok, "Mi chiamo Meg e ho 33 anni"} # A map of bindings is supported as well. assert Backend.lgettext("it", "interpolations", nil, "Hello %{name}", %{name: "Jane"}) == {:ok, "Ciao Jane"} end test "supports interpolation with missing translations" do msgid = "Hello %{name}, missing message!" assert Backend.lgettext("pl", "foo", nil, msgid, %{name: "Samantha"}) == {:default, "Hello Samantha, missing message!"} msgid = "Hello world!" assert Backend.lgettext("pl", "foo", nil, msgid, %{}) == {:default, "Hello world!"} msgid = "Hello %{name}" assert Backend.lgettext("pl", "foo", nil, msgid, %{}) == {:missing_bindings, "Hello %{name}", [:name]} end test "falls back to handle_missing_translation" do msgctxt = "some context" msgid = "Hello %{name}" bindings = %{name: "Jane"} assert Backend.lgettext("pl", "foo", msgctxt, msgid, bindings) == {:default, "Hello Jane"} assert_receive {"pl", "foo", ^msgctxt, ^msgid, ^bindings} end test "preserves the key when using the default c:handle_missing_bindings/2" do msgid = "My name is %{name} and I'm %{age}" assert Backend.lgettext("it", "interpolations", nil, msgid, %{name: "José"}) == {:missing_bindings, "Mi chiamo José e ho %{age} anni", [:age]} end test "strings are concatenated before generating function clauses" do msgid = "Concatenated and long string" assert Backend.lgettext("it", "default", msgid, %{}) == {:ok, "Stringa lunga e concatenata"} assert Backend.lgettext("it", "default", nil, msgid, %{}) == {:ok, "Stringa lunga e concatenata"} msgid = "A friend" msgid_plural = "%{count} friends" assert Backend.lngettext("it", "default", nil, msgid, msgid_plural, 1, %{}) == {:ok, "Un amico"} end test "warns if the domain contains slashes" do log = capture_log(fn -> assert Backend.lgettext("it", "sub/dir/domain", nil, "hello", %{}) == {:default, "hello"} end) assert log =~ ~s(Slashes in domains are not supported: "sub/dir/domain") end test "with :allowed_locales ignores other locales as strings" do assert GettextTest.BackendWithAllowedLocalesString.lgettext( "it", "default", nil, "Hello world", %{} ) == {:default, "Hello world"} assert GettextTest.BackendWithAllowedLocalesString.lgettext( "es", "default", nil, "Hello world", %{} ) == {:ok, "Hola mundo"} end test "with :allowed_locales ignores other locales as atom" do assert GettextTest.BackendWithAllowedLocalesAtom.lgettext( "it", "default", nil, "Hello world", %{} ) == {:default, "Hello world"} assert GettextTest.BackendWithAllowedLocalesAtom.lgettext( "es", "default", nil, "Hello world", %{} ) == {:ok, "Hola mundo"} end end describe "c:lngettext/7" do test "returns {:ok, translation} for found translations" do message = Backend.lngettext( "it", "errors", nil, "There was an error", "There were %{count} errors", 1, %{} ) assert message == {:ok, "C'è stato un errore"} message = Backend.lngettext( "it", "errors", nil, "There was an error", "There were %{count} errors", 3, %{} ) assert message == {:ok, "Ci sono stati 3 errori"} assert {:ok, "3 エラーがありました"} = Backend.lngettext( "ja", "errors", nil, "There was an error", "There were %{count} errors", 3, %{} ) end test "returns {:default, msgid(_plural)} for missing translations" do assert Backend.lngettext("it", "not a domain", nil, "foo", "foos", 1, %{}) == {:default, "foo"} assert Backend.lngettext("it", "not a domain", nil, "foo", "foos", 10, %{}) == {:default, "foos"} end test "returns {:default, msgid(_plural)} for translations with empty msgstr" do msgid = "Not even one msgstr" msgid_plural = "Not even %{count} msgstrs" assert Backend.lngettext("it", "default", nil, msgid, msgid_plural, 1, %{}) == {:default, "Not even one msgstr"} assert Backend.lngettext("it", "default", nil, msgid, msgid_plural, 2, %{}) == {:default, "Not even 2 msgstrs"} end test "supports interpolation" do msgid = "There was an error" msgid_plural = "There were %{count} errors" assert Backend.lngettext("it", "errors", nil, msgid, msgid_plural, 1, %{}) == {:ok, "C'è stato un errore"} assert Backend.lngettext("it", "errors", nil, msgid, msgid_plural, 4, %{}) == {:ok, "Ci sono stati 4 errori"} msgid = "You have one message, %{name}" msgid_plural = "You have %{count} messages, %{name}" assert Backend.lngettext("it", "interpolations", nil, msgid, msgid_plural, 1, %{ name: "Jane" }) == {:ok, "Hai un messaggio, Jane"} assert Backend.lngettext("it", "interpolations", nil, msgid, msgid_plural, 0, %{ name: "Jane" }) == {:ok, "Hai 0 messaggi, Jane"} end test "supports interpolation with missing translations" do msgid = "One error" msgid_plural = "%{count} errors" assert Backend.lngettext("pl", "foo", nil, msgid, msgid_plural, 1, %{}) == {:default, "One error"} assert Backend.lngettext("pl", "foo", nil, msgid, msgid_plural, 9, %{}) == {:default, "9 errors"} end test "falls back to c:handle_missing_binding/2" do msgid = "You have one message, %{name}" msgid_plural = "You have %{count} messages, %{name}" assert Backend.lngettext("it", "interpolations", nil, msgid, msgid_plural, 1, %{}) == {:missing_bindings, "Hai un messaggio, %{name}", [:name]} assert Backend.lngettext("it", "interpolations", nil, msgid, msgid_plural, 6, %{}) == {:missing_bindings, "Hai 6 messaggi, %{name}", [:name]} end test "uses the default c:handle_missing_plural_translation/7 implementation" do msgctxt = "some context" msgid = "Hello %{name}" msgid_plural = "Hello %{name}" bindings = %{name: "Jane"} assert Backend.lngettext( "pl", "foo", msgctxt, msgid, msgid_plural, 4, bindings ) == {:default, "Hello Jane"} assert_receive {"pl", "foo", ^msgctxt, ^msgid, ^msgid_plural, 4, ^bindings} end test "warns if the domain contains slashes" do log = capture_log(fn -> assert Backend.lngettext("it", "sub/dir/domain", nil, "hello", "hellos", 2, %{}) == {:default, "hellos"} end) assert log =~ ~s(Slashes in domains are not supported: "sub/dir/domain") end test "supports a custom Gettext.Plural module" do assert BackendWithCustomPluralForms.lngettext( "it", "default", nil, "One new email", "%{count} new emails", 1, %{} ) == {:ok, "1 nuove email"} assert BackendWithCustomPluralForms.lngettext( "it", "default", nil, "One new email", "%{count} new emails", 2, %{} ) == {:ok, "Una nuova email"} end test "supports a custom Gettext.Plural module with the context parameter" do alias BackendWithCustomCompiledPluralForms, as: T assert T.lngettext("it", "default", nil, "One new email", "%{count} new emails", 1, %{}) assert_received {:plural_context, %{plural_forms_header: "nplurals=2; plural=(n != 1);"}} end test "supports a custom Gettext.Plural module from app environment" do Application.put_env(:gettext, :plural_forms, GettextTest.CustomPlural) defmodule BackendWithAppPluralForms do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/single_messages" end assert BackendWithAppPluralForms.lngettext( "it", "default", nil, "One new email", "%{count} new emails", 1, %{} ) == {:ok, "1 nuove email"} assert BackendWithAppPluralForms.lngettext( "it", "default", nil, "One new email", "%{count} new emails", 2, %{} ) == {:ok, "Una nuova email"} after Application.put_env(:gettext, :plural_forms, Gettext.Plural) end test "raises an error if a plural message has no plural form for the given locale" do log = capture_log(fn -> Code.eval_quoted( quote do defmodule BadTranslations do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/bad_messages" end end ) end) assert log =~ "message is missing plural form 2 which is required by the locale \"ru\"" msgid = "should be at least %{count} character(s)" msgid_plural = "should be at least %{count} character(s)" assert_raise Gettext.PluralFormError, ~r/plural form 2 is required for locale \"ru\" but is missing/, fn -> # Dynamic module to avoid warnings. apply(BadTranslations, :lngettext, [ "ru", "errors", nil, msgid, msgid_plural, 8, %{} ]) end end end end ================================================ FILE: test/gettext/extractor_test.exs ================================================ defmodule Gettext.ExtractorTest do use ExUnit.Case import ExUnit.CaptureLog alias Expo.Message alias Expo.Messages alias Gettext.Extractor describe "merge_pot_files/2" do @tag :tmp_dir test "merges two POT files", %{tmp_dir: tmp_dir} do paths = %{ tomerge: Path.join(tmp_dir, "tomerge.pot"), ignored: Path.join(tmp_dir, "ignored.pot"), new: Path.join(tmp_dir, "new.pot") } extracted_po_structs = [ {paths.tomerge, %Messages{messages: [%Message.Singular{msgid: ["other"], msgstr: [""]}]}}, {paths.new, %Messages{messages: [%Message.Singular{msgid: ["new"], msgstr: [""]}]}} ] write_file(paths.tomerge, """ msgid "foo" msgstr "" """) write_file(paths.ignored, """ msgid "ignored" msgstr "" """) structs = Extractor.merge_pot_files(extracted_po_structs, [paths.tomerge, paths.ignored], []) # Unchanged files are not returned assert {_path, :unchanged} = List.keyfind(structs, paths.ignored, 0) assert {_, {:changed, contents}} = List.keyfind(structs, paths.tomerge, 0) assert IO.iodata_to_binary(contents) == """ msgid "foo" msgstr "" msgid "other" msgstr "" """ assert {_, {:changed, contents}} = List.keyfind(structs, paths.new, 0) contents = IO.iodata_to_binary(contents) assert contents =~ """ msgid "new" msgstr "" """ end @tag :tmp_dir test "reports the filename if syntax error", %{tmp_dir: tmp_dir} do path = Path.join(tmp_dir, "syntax_error.pot") write_file(path, """ msgid "foo" msgid "bar" msgstr "" """) message = ~r/syntax_error\.pot:3: syntax error before: msgid/ assert_raise Expo.PO.SyntaxError, message, fn -> Extractor.merge_pot_files([{path, %Messages{messages: []}}], [path], []) end end end describe "merge_template/2" do test "non-autogenerated messages are kept" do # No autogenerated messages message_1 = %Message.Singular{msgid: ["foo"], msgstr: ["bar"]} message_2 = %Message.Singular{msgid: ["baz"], msgstr: ["bong"]} message_3 = %Message.Singular{msgid: ["a", "b"], msgstr: ["c", "d"]} old = %Messages{messages: [message_1]} new = %Messages{messages: [message_2, message_3]} assert Extractor.merge_template(old, new, []) == %Messages{ messages: [message_1, message_2, message_3] } end test "allowed messages are kept" do message_1 = %Message.Singular{ msgid: ["foo"], msgstr: ["bar"], references: [[{"foo.ex", 1}, {"bar.ex", 1}], [{"baz.ex", 1}]], flags: [["elixir-autogen", "elixir-format"]] } message_2 = %Message.Singular{ msgid: ["baz"], msgstr: ["bong"], references: [{"web/static/js/app.js", 10}] } old = %Messages{messages: [message_1, message_2]} new = %Messages{messages: []} assert Extractor.merge_template(old, new, excluded_refs_from_purging: ~r{^web/static/}) == %Messages{messages: [message_2]} end test "obsolete autogenerated messages are discarded" do # Autogenerated messages message_1 = %Message.Singular{ msgid: ["foo"], msgstr: ["bar"], flags: [["elixir-autogen", "elixir-format"]] } message_2 = %Message.Singular{msgid: ["baz"], msgstr: ["bong"]} old = %Messages{messages: [message_1]} new = %Messages{messages: [message_2]} assert Extractor.merge_template(old, new, []) == %Messages{messages: [message_2]} end test "matching messages are merged" do ts1 = [ %Message.Singular{ msgid: ["matching autogenerated"], references: [{"foo.ex", 2}], flags: [["elixir-autogen"]], extracted_comments: ["#. Foo"] }, %Message.Singular{msgid: ["non-matching autogenerated"], flags: [["elixir-autogen"]]}, %Message.Singular{msgid: ["non-autogenerated"], references: [{"foo.ex", 4}]} ] ts2 = [ %Message.Singular{msgid: ["non-matching non-autogenerated"]}, %Message.Plural{ msgid: ["matching autogenerated"], msgid_plural: ["matching non-autogenerated 2"], references: [{"foo.ex", 3}], extracted_comments: ["#. Bar"], flags: [["elixir-autogen"]] } ] assert Extractor.merge_template( %Messages{messages: ts1}, %Messages{messages: ts2}, [] ) == %Messages{ messages: [ %Message.Plural{ msgid: ["matching autogenerated"], msgid_plural: ["matching non-autogenerated 2"], references: [{"foo.ex", 3}], flags: [["elixir-autogen"]], extracted_comments: ["#. Bar"] }, %Message.Singular{ msgid: ["non-autogenerated"], references: [{"foo.ex", 4}] }, %Message.Singular{msgid: ["non-matching non-autogenerated"]} ] } end test "headers are taken from the oldest PO file" do po1 = %Messages{ headers: ["Last-Translator: Foo", "Content-Type: text/plain"], messages: [] } po2 = %Messages{headers: ["Last-Translator: Bar"], messages: []} assert Extractor.merge_template(po1, po2, []) == %Messages{ headers: [ "Last-Translator: Foo", "Content-Type: text/plain" ], messages: [] } end test "non-empty msgstrs raise an error" do po1 = %Messages{messages: [%Message.Singular{msgid: ["foo"], msgstr: ["bar"]}]} po2 = %Messages{messages: [%Message.Singular{msgid: ["foo"], msgstr: ["bar"]}]} msg = "message with msgid 'foo' has a non-empty msgstr" assert_raise Gettext.Error, msg, fn -> Extractor.merge_template(po1, po2, []) end end test "order is kept as much as possible" do # Old messages are kept in the order we find them (except the ones we # remove), and all the new ones are appended after them. foo_message = %Message.Singular{msgid: ["foo"], references: [{"foo.ex", 1}]} msgid = "Live stream available from %{provider}" po1 = %Messages{ messages: [ %Message.Singular{msgid: [msgid], references: [{"reminder.ex", 160}]}, foo_message ] } po2 = %Messages{ messages: [ %Message.Singular{msgid: ["new message"]}, foo_message, %Message.Singular{msgid: [msgid], references: [{"live_streaming.ex", 40}]} ] } %Messages{messages: [message_1, ^foo_message, message_2]} = Extractor.merge_template(po1, po2, []) assert message_1.msgid == [msgid] assert message_1.references == [{"live_streaming.ex", 40}] assert message_2.msgid == ["new message"] end test "messages can be ordered alphabetically through the :sort_by_msgid option" do # Old and new messages are mixed together and ordered alphabetically. foo_message_uppercase = %Message.Singular{msgid: ["FOO"], references: [{"FOO.ex", 1}]} foo_message = %Message.Singular{msgid: ["", "foo"], references: [{"foo.ex", 1}]} bar_message = %Message.Singular{msgid: ["ba", "r"], references: [{"bar.ex", 1}]} baz_message = %Message.Plural{ msgid: ["b", "az"], msgid_plural: ["bazs"], references: [{"baz.ex", 1}] } qux_message = %Message.Singular{msgid: ["qux", ""], references: [{"bar.ex", 1}]} po1 = %Messages{ messages: [ foo_message_uppercase, foo_message, qux_message, bar_message ] } po2 = %Messages{ messages: [ baz_message, foo_message, bar_message, foo_message_uppercase ] } %Messages{messages: messages} = Extractor.merge_template(po1, po2, sort_by_msgid: :case_sensitive) assert Enum.map(messages, &IO.chardata_to_string(&1.msgid)) == ~w(FOO bar baz foo qux) end test "messages can be ordered alphabetically through the :sort_by_msgid_case_insensitive option" do # Old and new messages are mixed together and ordered alphabetically in a case insensitive fashion. foo_1_message = %Message.Singular{msgid: ["foo"], references: [{"foo.ex", 1}]} foo_2_message = %Message.Singular{msgid: ["Foo"], references: [{"Foo.ex", 1}]} foo_3_message = %Message.Singular{msgid: ["FOO"], references: [{"FOO.ex", 1}]} bar_message = %Message.Singular{msgid: ["bar"], references: [{"bar.ex", 1}]} qux_message = %Message.Singular{msgid: ["qux"], references: [{"qux.ex", 1}]} po1 = %Messages{ messages: [ foo_1_message, qux_message, foo_2_message, bar_message, foo_3_message ] } po2 = %Messages{ messages: [ bar_message, foo_1_message, bar_message ] } %Messages{messages: messages} = Extractor.merge_template(po1, po2, sort_by_msgid: :case_insensitive) assert Enum.map(messages, &IO.chardata_to_string(&1.msgid)) == ~w(bar foo Foo FOO qux) end end test "extraction process" do refute Extractor.extracting?() Extractor.enable() assert Extractor.extracting?() code = """ defmodule Gettext.ExtractorTest.MyGettext do use Gettext.Backend, otp_app: :test_application end defmodule Gettext.ExtractorTest.MyOtherGettext do use Gettext.Backend, otp_app: :test_application, priv: "messages" end defmodule Foo do require Gettext.Macros def bar do Gettext.Macros.gettext_comment("some comment") Gettext.Macros.gettext_comment("some other comment") Gettext.Macros.gettext_comment("repeated comment") Gettext.Macros.gettext_with_backend(Gettext.ExtractorTest.MyGettext, "foo") Gettext.Macros.dngettext_with_backend(Gettext.ExtractorTest.MyGettext, "errors", "one error", "%{count} errors", 2) Gettext.Macros.dngettext_with_backend(Gettext.ExtractorTest.MyGettext, "errors", "one error", "%{count} errors", 2) Gettext.Macros.dgettext_with_backend(Gettext.ExtractorTest.MyGettext, "errors", "one error") Gettext.Macros.gettext_comment("one more comment") Gettext.Macros.gettext_comment("repeated comment") Gettext.Macros.gettext_comment("repeated comment") Gettext.Macros.gettext_with_backend(Gettext.ExtractorTest.MyGettext, "foo") Gettext.Macros.dgettext_with_backend(Gettext.ExtractorTest.MyOtherGettext, "greetings", "hi") Gettext.Macros.pgettext_with_backend(Gettext.ExtractorTest.MyGettext, "test", "context based message") end end """ Code.compile_string(code, Path.join(File.cwd!(), "foo.ex")) expected = [ {"priv/gettext/default.pot", ~S""" msgid "" msgstr "" #. some comment #. some other comment #. repeated comment #. one more comment #: foo.ex:16 #: foo.ex:23 #, elixir-autogen, elixir-format msgid "foo" msgstr "" #: foo.ex:25 #, elixir-autogen, elixir-format msgctxt "test" msgid "context based message" msgstr "" """}, {"priv/gettext/errors.pot", ~S""" msgid "" msgstr "" #: foo.ex:17 #: foo.ex:18 #: foo.ex:19 #, elixir-autogen, elixir-format msgid "one error" msgid_plural "%{count} errors" msgstr[0] "" msgstr[1] "" """}, {"messages/greetings.pot", ~S""" msgid "" msgstr "" #: foo.ex:24 #, elixir-autogen, elixir-format msgid "hi" msgstr "" """} ] # No backends for the unknown app assert [] = Extractor.pot_files(:unknown, []) pot_files = Extractor.pot_files(:test_application, []) dumped = pot_files |> Enum.reject(&match?({_path, :unchanged}, &1)) |> Enum.map(fn {k, {:changed, v}} -> {k, IO.iodata_to_binary(v)} end) # We check that dumped strings end with the `expected` string because # there's the informative comment at the start of each dumped string. Enum.each(dumped, fn {path, contents} -> {^path, expected_contents} = List.keyfind(expected, path, 0) assert String.starts_with?(contents, "## This file is a PO Template file.") assert contents =~ expected_contents end) after Extractor.disable() refute Extractor.extracting?() end test "warns on conflicting backends" do refute Extractor.extracting?() Extractor.enable() assert Extractor.extracting?() code = """ defmodule Gettext.ExtractorConflictTest.MyGettext do use Gettext.Backend, otp_app: :test_application end defmodule Gettext.ExtractorConflictTest.MyOtherGettext do use Gettext.Backend, otp_app: :test_application end defmodule FooConflict do require Gettext.Macros def bar do Gettext.Macros.gettext_with_backend(Gettext.ExtractorConflictTest.MyGettext, "foo") Gettext.Macros.gettext_with_backend(Gettext.ExtractorConflictTest.MyOtherGettext, "foo") end end """ assert ExUnit.CaptureIO.capture_io(:stderr, fn -> Code.compile_string(code, Path.join(File.cwd!(), "foo_conflict.ex")) Extractor.pot_files(:test_application, []) end) =~ "the Gettext backend Gettext.ExtractorConflictTest.MyGettext has the same :priv directory as Gettext.ExtractorConflictTest.MyOtherGettext" after Extractor.disable() end test "warns on conflicting plural messages" do refute Extractor.extracting?() Extractor.enable() assert Extractor.extracting?() code = """ defmodule Gettext.ExtractorTest.ConflictingPlural.Gettext do use Gettext.Backend, otp_app: :test_conflicting_plural end defmodule Gettext.ExtractorTest.ConflictingPlural.Foo do require Gettext.Macros def bar do Gettext.Macros.dngettext_with_backend(Gettext.ExtractorTest.ConflictingPlural.Gettext, "errors", "one error", "%{count} errors", 2) Gettext.Macros.dngettext_with_backend(Gettext.ExtractorTest.ConflictingPlural.Gettext, "errors", "one error", "multiple errors", 2) end end """ assert capture_log(fn -> Code.compile_string(code, Path.join(File.cwd!(), "foo.ex")) end) =~ """ Plural message for 'one error' is not matching: Using 'multiple errors' instead of '%{count} errors'. References: foo.ex:9, foo.ex:10 """ after Extractor.disable() end defp write_file(path, contents) do path |> Path.dirname() |> File.mkdir_p!() File.write!(path, contents) end end ================================================ FILE: test/gettext/fuzzy_test.exs ================================================ defmodule Gettext.FuzzyTest do use ExUnit.Case, async: true alias Gettext.Fuzzy alias Expo.Message test "matcher/1" do assert Fuzzy.matcher(0.5).({nil, "foo"}, {nil, "foo"}) == {:match, 1.0} assert Fuzzy.matcher(0.5).({nil, "foo"}, {nil, "bar"}) == :nomatch assert Fuzzy.matcher(0.0).({nil, "foo"}, {nil, "bar"}) == {:match, 0.0} end describe "jaro_distance/2" do test "compares the distance of the msgid" do assert Fuzzy.jaro_distance({nil, "foo"}, {nil, "foo"}) == 1.0 assert Fuzzy.jaro_distance({nil, "foo"}, {nil, "foos"}) > 0.0 assert Fuzzy.jaro_distance({nil, "foo"}, {nil, "bar"}) == 0.0 end test "with one message and one plural message, only the msgids are compared" do assert Fuzzy.jaro_distance({nil, "foo"}, {nil, {"foo", "bar"}}) == 1.0 assert Fuzzy.jaro_distance({nil, {"foo", "bar"}}, {nil, "foo"}) == 1.0 end test "completely ignores the msgctxt in the key when calculating the distance" do assert Fuzzy.jaro_distance({"a", "foo"}, {"b", "foo"}) == 1.0 assert Fuzzy.jaro_distance({"same", "foo"}, {"same", "bar"}) == 0.0 end end describe "merge/2" do test "two messages" do message_1 = %Message.Singular{msgid: ["foo"]} message_2 = %Message.Singular{msgid: ["foos"], msgstr: ["bar"]} assert %Message.Singular{} = message = Fuzzy.merge(message_1, message_2) assert message.msgid == ["foo"] assert message.msgstr == ["bar"] assert Message.has_flag?(message, "fuzzy") end test "a message and a plural message" do message_1 = %Message.Singular{msgid: ["foo"]} message_2 = %Message.Plural{ msgid: ["foos"], msgid_plural: ["bar"], msgstr: %{0 => ["a"], 1 => ["b"]} } assert %Message.Singular{} = message = Fuzzy.merge(message_1, message_2) assert message.msgid == ["foo"] assert message.msgstr == ["a"] assert Message.has_flag?(message, "fuzzy") end test "a plural message and a message" do message_1 = %Message.Plural{ msgid: ["foos"], msgid_plural: ["bar"], msgstr: %{0 => [], 1 => []} } message_2 = %Message.Singular{msgid: ["foo"], msgstr: ["bar"]} assert %Message.Plural{} = message = Fuzzy.merge(message_1, message_2) assert message.msgid == ["foos"] assert message.msgid_plural == ["bar"] assert message.msgstr == %{0 => ["bar"], 1 => ["bar"]} assert Message.has_flag?(message, "fuzzy") end test "two plural messages" do message_1 = %Message.Plural{msgid: ["foos"], msgid_plural: ["bar"]} message_2 = %Message.Plural{ msgid: ["foo"], msgid_plural: ["baz"], msgstr: %{0 => ["a"], 1 => ["b"]} } assert %Message.Plural{} = message = Fuzzy.merge(message_1, message_2) assert message.msgid == ["foos"] assert message.msgid_plural == ["bar"] assert message.msgstr == %{0 => ["a"], 1 => ["b"]} assert Message.has_flag?(message, "fuzzy") end end end ================================================ FILE: test/gettext/interpolation/default_test.exs ================================================ defmodule Gettext.Interpolation.DefaultTest do use ExUnit.Case, async: true doctest Gettext.Interpolation.Default alias Gettext.Interpolation.Default, as: Interpolation require Interpolation test "runtime_interpolate/2" do interpolatable = Interpolation.to_interpolatable("%{a} %{b} %{c}") assert Interpolation.runtime_interpolate(interpolatable, %{a: 1, b: :two, c: "thr ee"}) == {:ok, "1 two thr ee"} assert Interpolation.runtime_interpolate(interpolatable, %{a: "a"}) == {:missing_bindings, "a %{b} %{c}", [:b, :c]} interpolatable = Interpolation.to_interpolatable("%{a} %{a} %{a}") assert Interpolation.runtime_interpolate(interpolatable, %{a: "foo"}) == {:ok, "foo foo foo"} assert Interpolation.runtime_interpolate(interpolatable, %{b: "bar"}) == {:missing_bindings, "%{a} %{a} %{a}", [:a]} assert Interpolation.runtime_interpolate("%{a} %{b} %{c}", %{a: "a"}) == {:missing_bindings, "a %{b} %{c}", [:b, :c]} end test "to_interpolatable/1" do assert Interpolation.to_interpolatable("Hello %{name}") == ["Hello ", :name] assert Interpolation.to_interpolatable("%{solo}") == [:solo] assert Interpolation.to_interpolatable("%{foo}%{bar} %{baz}") == [:foo, :bar, " ", :baz] assert Interpolation.to_interpolatable("%{Your name} is cool!") == [:"Your name", " is cool!"] assert Interpolation.to_interpolatable("foo %{} bar") == ["foo %{} bar"] assert Interpolation.to_interpolatable("%{") == ["%{"] assert Interpolation.to_interpolatable("abrupt ending %{") == ["abrupt ending %{"] assert Interpolation.to_interpolatable("incomplete %{ and then some") == ["incomplete %{ and then some"] assert Interpolation.to_interpolatable("first empty %{} then %{ incomplete") == ["first empty %{} then %{ incomplete"] assert Interpolation.to_interpolatable("") == [] end if System.otp_release() >= "20" do test "to_interpolatable/1 with Unicode" do assert Interpolation.to_interpolatable("%{Héllø} there") == [String.to_atom("Héllø"), " there"] end end test "keys/1" do # With a string as its argument assert Interpolation.keys("Hello %{name}") == [:name] assert Interpolation.keys("It's %{time} here in %{state}") == [:time, :state] assert Interpolation.keys("Hi there %{your name}") == [:"your name"] assert Interpolation.keys("Hello %{name} in %{state} goodbye %{name}") == [:name, :state] # With a list of segments as its argument assert Interpolation.keys(["Hello ", :name, " it's ", :time, " goodbye ", :name]) == [:name, :time] end describe "compile_interpolate/3" do test "interpolates complete bindings" do assert {:ok, "Hello World!"} == Interpolation.compile_interpolate(:translation, "Hello %{name}!", %{name: "World"}) end test "interpolates incomplete bindings" do assert {:missing_bindings, "Hello %{name}!", [:name]} == Interpolation.compile_interpolate(:translation, "Hello %{name}!", %{ unused: "binding" }) end test "interpolates no bindings" do assert {:missing_bindings, "Hello %{name}!", [:name]} == Interpolation.compile_interpolate(:translation, "Hello %{name}!", %{}) end test "rejects dynamic message" do assert_raise RuntimeError, fn -> Code.eval_quoted( quote do require Interpolation Interpolation.compile_interpolate( :translation, "dynamic message " <> inspect(make_ref()), %{} ) end ) end end test "optimizes plural message without count" do translate = fn bindings -> Interpolation.compile_interpolate( :plural_translation, "%{count} shoes", bindings ) end assert_raise MatchError, fn -> translate.(%{}) end assert {:ok, "7 shoes"} = translate.(%{count: 7}) end end end ================================================ FILE: test/gettext/macros_test.exs ================================================ defmodule Gettext.MacrosTest.Translator do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/single_messages" end defmodule Gettext.MacrosTest do use ExUnit.Case, async: true use Gettext, backend: Gettext.MacrosTest.Translator import ExUnit.CaptureLog require Gettext.Macros, as: Macros @backend Gettext.MacrosTest.Translator @gettext_msgid "Hello world" describe "gettext/1" do test "supports binary-ish msgid at compile-time" do Gettext.put_locale(@backend, "it") assert gettext("Hello world") == "Ciao mondo" assert gettext(@gettext_msgid) == "Ciao mondo" assert gettext(~s(Hello world)) == "Ciao mondo" assert gettext("Hello " <> "world") == "Ciao mondo" assert gettext("Hello " <> "wor" <> "ld") == "Ciao mondo" assert gettext(<<"Hello world">>) == "Ciao mondo" assert gettext(<<"Hello ", "world">>) == "Ciao mondo" assert gettext("Hello " <> <<"wor", "ld">>) == "Ciao mondo" assert gettext("Hello " <> ~s(world)) == "Ciao mondo" assert gettext(~S(Hello ) <> ~s(world)) == "Ciao mondo" end end describe "dgettext/3" do test "supports binary-ish msgid at compile-time" do Gettext.put_locale(@backend, "it") assert dgettext("errors", "Invalid email address") == "Indirizzo email non valido" keys = %{name: "Jim"} assert dgettext("interpolations", "Hello %{name}", keys) == "Ciao Jim" log = capture_log(fn -> assert dgettext("interpolations", "Hello %{name}") == "Ciao %{name}" end) assert log =~ ~s/[error] missing Gettext bindings: [:name]/ end end describe "pgettext/3" do test "supports test with context based messages" do Gettext.put_locale(@backend, "it") assert pgettext("test", @gettext_msgid) == "Ciao mondo" assert pgettext("test", ~s(Hello world)) == "Ciao mondo" assert pgettext("test", "Hello world", %{}) == "Ciao mondo" assert pgettext("test", "Hello %{name}", %{name: "Marco"}) == "Ciao Marco" # Missing message assert pgettext("test", "Hello missing", %{}) == "Hello missing" end end test "pgettext/3, pngettext/4: dynamic context raises" do error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator context = "test" pgettext(context, "Hello world") end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:context" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator context = "test" pngettext(context, "Hello world", "Hello world", 5) end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:context" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" end test "dpgettext/4, dpngettext/5: dynamic context or dynamic domain raises" do error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator context = "test" dpgettext("default", context, "Hello world") end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:context" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator domain = "test" dpgettext(domain, "test", "Hello world") end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:domain" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator context = "test" dpngettext("default", context, "Hello world", "Hello world", n) end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:context" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator context = "test" dpngettext(domain, "test", "Hello world", "Hello World", n) end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:domain" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" end test "dpgettext/4: context and domain based messages" do Gettext.put_locale(@backend, "it") assert dpgettext("default", "test", "Hello world", %{}) == "Ciao mondo" end test "dgettext/3 and dngettext/2: non-binary things at compile-time" do error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator context = "test" dgettext("errors", msgid) end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:msgid" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator msgid_plural = ~s(foo #{1 + 1} bar) dngettext("default", "foo", msgid_plural, 1) end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:msgid_plural" assert message =~ "Gettext.gettext(Gettext.MacrosTest.Translator, string)" error = assert_raise ArgumentError, fn -> defmodule Sample do use Gettext, backend: Gettext.MacrosTest.Translator domain = "dynamic_domain" dgettext(domain, "hello") end end message = ArgumentError.message(error) assert message =~ "Gettext macros expect message keys" assert message =~ "{:domain" end describe "dngettext/5" do test "translates with plural and domain" do Gettext.put_locale(@backend, "it") assert dngettext( "interpolations", "You have one message, %{name}", "You have %{count} messages, %{name}", 1, %{name: "James"} ) == "Hai un messaggio, James" assert dngettext( "interpolations", "You have one message, %{name}", "You have %{count} messages, %{name}", 2, %{name: "James"} ) == "Hai 2 messaggi, James" assert dngettext( "interpolations", "Month", "%{count} months", 2 ) == "2 mesi" end end @ngettext_msgid "One new email" @ngettext_msgid_plural "%{count} new emails" describe "ngettext/4" do test "translates with plural" do Gettext.put_locale(@backend, "it") assert ngettext("One new email", "%{count} new emails", 1) == "Una nuova email" assert ngettext("One new email", "%{count} new emails", 2) == "2 nuove email" assert ngettext(@ngettext_msgid, @ngettext_msgid_plural, 1) == "Una nuova email" assert ngettext(@ngettext_msgid, @ngettext_msgid_plural, 2) == "2 nuove email" end end describe "pngettext/4" do test "translates with plurals and context" do Gettext.put_locale(@backend, "it") assert pngettext("test", "One new email", "%{count} new emails", 1) == "Una nuova test email" assert pngettext("test", "One new email", "%{count} new emails", 2) == "2 nuove test email" assert pngettext("test", @ngettext_msgid, @ngettext_msgid_plural, 1) == "Una nuova test email" assert pngettext("test", @ngettext_msgid, @ngettext_msgid_plural, 2) == "2 nuove test email" end end test "the d?n?gettext macros support a kw list for interpolation" do Gettext.put_locale(@backend, "it") assert gettext("%{msg}", msg: "foo") == "foo" end test "(d)(p)gettext_noop" do assert dpgettext_noop("errors", "test", "Oops") == "Oops" assert dgettext_noop("errors", "Oops") == "Oops" assert gettext_noop("Hello %{name}!") == "Hello %{name}!" end test "(d)(p)ngettext_noop" do assert dpngettext_noop("errors", "test", "One error", "%{count} errors") == {"One error", "%{count} errors"} assert dngettext_noop("errors", "One error", "%{count} errors") == {"One error", "%{count} errors"} assert ngettext_noop("One message", "%{count} messages") == {"One message", "%{count} messages"} assert pngettext_noop("test", "One message", "%{count} messages") == {"One message", "%{count} messages"} end ## _with_backend variants describe "dpgettext_noop_with_backend/4" do test "supports test with context based messages" do assert Macros.dpgettext_noop_with_backend(@backend, "test", "ctx", "Hello world") == "Hello world" end end describe "dgettext_noop_with_backend/4" do test "supports test with context based messages" do assert Macros.dgettext_noop_with_backend(@backend, "test", "Hello world") == "Hello world" end end describe "pgettext_noop_with_backend/4" do test "supports test with context based messages" do assert Macros.pgettext_noop_with_backend(@backend, "ctx", "Hello world") == "Hello world" end end describe "gettext_noop_with_backend/2" do test "supports test with context based messages" do assert Macros.gettext_noop_with_backend(@backend, "Hello world") == "Hello world" end end describe "dpngettext_noop_with_backend/5" do test "supports test with context based messages" do assert Macros.dpngettext_noop_with_backend( @backend, "test", "ctx", "One message", "%{count} messages" ) == {"One message", "%{count} messages"} end end describe "dngettext_noop_with_backend/4" do test "supports test with context based messages" do assert Macros.dngettext_noop_with_backend( @backend, "test", "One message", "%{count} messages" ) == {"One message", "%{count} messages"} end end describe "pngettext_noop_with_backend/4" do test "supports test with context based messages" do assert Macros.pngettext_noop_with_backend( @backend, "ctx", "One message", "%{count} messages" ) == {"One message", "%{count} messages"} end end describe "ngettext_noop_with_backend/3" do test "supports test with context based messages" do assert Macros.ngettext_noop_with_backend(@backend, "One message", "%{count} messages") == {"One message", "%{count} messages"} end end describe "translation macros *_with_backend" do setup do Gettext.put_locale(@backend, "it") :ok end test "dpgettext_with_backend/5" do assert Macros.dpgettext_with_backend(@backend, "default", "test", "Hello world", %{}) == "Ciao mondo" end test "dgettext_with_backend/4" do assert Macros.dgettext_with_backend(@backend, "default", "Hello world") == "Ciao mondo" end test "pgettext_with_backend/4" do assert Macros.pgettext_with_backend(@backend, "test", "Hello world") == "Ciao mondo" end test "gettext_with_backend/2" do assert Macros.gettext_with_backend(@backend, "Hello world") == "Ciao mondo" end test "dpngettext_with_backend/6" do assert Macros.dpngettext_with_backend( @backend, "default", "test", "One new email", "%{count} new emails", 1 ) == "Una nuova test email" end test "dngettext_with_backend/5" do assert Macros.dngettext_with_backend( @backend, "default", "One new email", "%{count} new emails", 1 ) == "Una nuova email" end test "pngettext_with_backend/5" do assert Macros.pngettext_with_backend( @backend, "test", "One new email", "%{count} new emails", 1 ) == "Una nuova test email" end test "ngettext_with_backend/4" do assert Macros.ngettext_with_backend( @backend, "One new email", "%{count} new emails", 1 ) == "Una nuova email" end end end ================================================ FILE: test/gettext/merger_test.exs ================================================ defmodule Gettext.MergerTest do use ExUnit.Case, async: true import ExUnit.CaptureIO alias Expo.Message alias Expo.Messages alias Gettext.Merger @opts fuzzy: true, fuzzy_threshold: 0.8 @gettext_config [] @autogenerated_flags [["elixir-format"]] describe "merge/5" do test "headers from the old file are kept" do old_po = %Messages{ headers: [~S(Language: it\n), ~S(My-Header: my-value\n)], messages: [] } new_pot = %Messages{headers: ["foo"], messages: []} assert {new_po, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert new_po.headers == old_po.headers end test "obsolete messages are discarded (even the manually entered ones)" do old_po = %Messages{ messages: [ %Message.Singular{msgid: "obs_auto", msgstr: "foo", flags: @autogenerated_flags}, %Message.Singular{msgid: "obs_manual", msgstr: "foo"}, %Message.Singular{msgid: "tomerge", msgstr: "foo"} ] } new_pot = %Messages{messages: [%Message.Singular{msgid: "tomerge", msgstr: ""}]} assert {%Messages{messages: [message]}, stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.msgid == "tomerge" assert message.msgstr == "foo" assert stats == %{ exact_matches: 1, fuzzy_matches: 0, new: 0, removed: 2, marked_as_obsolete: 0 } end test "obsolete messages are marked as obsolete" do old_po = %Messages{ messages: [ %Message.Singular{msgid: "obs_auto", msgstr: "foo", flags: @autogenerated_flags}, %Message.Singular{msgid: "obs_manual", msgstr: "foo"}, %Message.Singular{msgid: "tomerge", msgstr: "foo", obsolete: true} ] } new_pot = %Messages{messages: [%Message.Singular{msgid: "tomerge", msgstr: ""}]} assert {%Messages{ messages: [ %Message.Singular{msgid: "tomerge", obsolete: false}, %Message.Singular{msgid: "obs_auto", obsolete: true}, %Message.Singular{msgid: "obs_manual", obsolete: true} ] }, stats} = Merger.merge( old_po, new_pot, "en", @opts ++ [on_obsolete: :mark_as_obsolete], @gettext_config ) assert stats == %{ exact_matches: 1, fuzzy_matches: 0, new: 0, removed: 0, marked_as_obsolete: 2 } end test "when messages match, the msgstr of the old one is preserved" do old_po = %Messages{messages: [%Message.Singular{msgid: "foo", msgstr: "bar"}]} new_pot = %Messages{messages: [%Message.Singular{msgid: "foo", msgstr: ""}]} assert {%Messages{messages: [message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.msgid == "foo" assert message.msgstr == "bar" end test "when messages match, existing translator comments are preserved" do # Note that the new message *should* not have any translator comments # (comes from a POT file). old_po = %Messages{ messages: [ %Message.Singular{msgid: "foo", comments: ["# existing comment"]} ] } new_pot = %Messages{ messages: [ %Message.Singular{msgid: "foo", comments: ["# new comment"]} ] } assert {%Messages{messages: [message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.msgid == "foo" assert message.comments == ["# existing comment"] end test "when messages match, existing translator flags are preserved" do old_po = %Messages{ messages: [%Message.Singular{msgid: "foo", flags: [["fuzzy"]]}] } new_pot = %Messages{messages: [%Message.Singular{msgid: "foo"}]} assert {%Messages{messages: [message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert Message.has_flag?(message, "fuzzy") end test "when messages match, existing extracted comments are replaced by new ones" do old_po = %Messages{ messages: [ %Message.Singular{ msgid: "foo", extracted_comments: ["#. existing comment", "#. other existing comment"] } ] } new_pot = %Messages{ messages: [ %Message.Singular{msgid: "foo", extracted_comments: ["#. new comment"]} ] } assert {%Messages{messages: [message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.extracted_comments == ["#. new comment"] end test "when messages match, existing references are replaced by new ones" do old_po = %Messages{ messages: [ %Message.Singular{msgid: "foo", references: [{"foo.ex", 1}]} ] } new_pot = %Messages{ messages: [ %Message.Singular{msgid: "foo", references: [{"bar.ex", 1}]} ] } assert {%Messages{messages: [message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.references == [{"bar.ex", 1}] end test "when messages match, existing flags are replaced by new ones" do old_po = %Messages{messages: [%Message.Singular{msgid: "foo"}]} new_pot = %Messages{ messages: [ %Message.Singular{msgid: "foo", flags: @autogenerated_flags} ] } assert {%Messages{messages: [message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.flags == @autogenerated_flags end test "messages with same msgid but different msgctxt are completely different" do old_po = %Messages{ messages: [ %Message.Singular{msgid: "foo", msgstr: "no context"}, %Message.Singular{ msgid: "foo", msgctxt: "context", msgstr: "with context" } ] } new_pot = %Messages{ messages: [ %Message.Singular{msgid: "foo", msgctxt: "context", msgstr: ""}, %Message.Singular{msgid: "foo", msgctxt: "other context", msgstr: ""}, %Message.Singular{msgid: "foo", msgstr: ""} ] } assert {%Messages{messages: messages}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert [ %Message.Singular{msgid: "foo", msgctxt: "context", msgstr: "with context"}, %Message.Singular{msgid: "foo", msgctxt: "other context", msgstr: "no context"} = _fuzzy, %Message.Singular{msgid: "foo", msgstr: "no context"} ] = messages end test "new messages are fuzzy-matched against obsolete messages" do old_message = %Message.Singular{ msgid: ["hello world!"], msgstr: ["foo"], comments: ["# existing comment"], extracted_comments: ["#. existing comment"], references: [{"foo.ex", 1}] } old_po = %Messages{messages: [old_message]} new_pot = %Messages{ messages: [ %Message.Singular{ msgid: "hello worlds!", references: [{"foo.ex", 2}], extracted_comments: ["#. new comment"], flags: [["my-flag"]] } ] } assert {%Messages{messages: [message]}, stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert %{exact_matches: 0, fuzzy_matches: 1, new: 0, removed: 0} = stats assert message.msgid == "hello worlds!" assert message.msgstr == ["foo"] assert message.comments == ["# existing comment"] assert message.extracted_comments == ["#. new comment"] assert message.references == [{"foo.ex", 2}] assert message.flags == [["my-flag", "fuzzy"]] assert message.previous_messages == [] assert {%Messages{messages: [message]}, _stats} = Merger.merge( old_po, new_pot, "en", @opts ++ [store_previous_message_on_fuzzy_match: true], @gettext_config ) assert message.previous_messages == [old_message] end test "exact matches have precedence over fuzzy matches" do old_po = %Messages{ messages: [ %Message.Singular{msgid: ["hello world!"], msgstr: ["foo"]}, %Message.Singular{msgid: ["hello worlds!"], msgstr: ["bar"]} ] } new_pot = %Messages{ messages: [%Message.Singular{msgid: ["hello world!"]}] } # Let's check that the "hello worlds!" message is discarded even if it's # a fuzzy match for "hello world!". assert {%Messages{messages: [message]}, stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) refute Message.has_flag?(message, "fuzzy") assert message.msgid == ["hello world!"] assert message.msgstr == ["foo"] assert %{exact_matches: 1, fuzzy_matches: 0, new: 0, removed: 1} = stats end test "exact matches do not prevent fuzzy matches for other messages" do old_po = %Messages{ messages: [%Message.Singular{msgid: ["hello world"], msgstr: ["foo"]}] } # "hello world" will match exactly. # "hello world!" should still get a fuzzy match. new_pot = %Messages{ messages: [ %Message.Singular{msgid: ["hello world"]}, %Message.Singular{msgid: ["hello world!"]} ] } assert {%Messages{messages: [message_1, message_2]}, stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message_1.msgid == ["hello world"] assert message_1.msgstr == ["foo"] refute Message.has_flag?(message_1, "fuzzy") assert message_2.msgid == ["hello world!"] assert message_2.msgstr == ["foo"] assert Message.has_flag?(message_2, "fuzzy") assert stats.new == 0 assert stats.removed == 0 assert stats.fuzzy_matches == 1 assert stats.exact_matches == 1 end test "multiple messages can fuzzy match against a single message" do old_po = %Messages{ messages: [%Message.Singular{msgid: ["hello world"], msgstr: ["foo"]}] } new_pot = %Messages{ messages: [ %Message.Singular{msgid: ["hello world 1"]}, %Message.Singular{msgid: ["hello world 2"]} ] } assert {%Messages{messages: [message_1, message_2]}, stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message_1.msgid == ["hello world 1"] assert message_1.msgstr == ["foo"] assert Message.has_flag?(message_1, "fuzzy") assert message_2.msgid == ["hello world 2"] assert message_2.msgstr == ["foo"] assert Message.has_flag?(message_2, "fuzzy") assert %{exact_matches: 0, new: 0, fuzzy_matches: 2, removed: 0} = stats end test "filling in a fuzzy message preserves references" do old_po = %Messages{ messages: [ %Message.Singular{ msgid: ["hello world!"], msgstr: ["foo"], comments: ["# old comment"], references: [{"old_file.txt", 1}] } ] } new_pot = %Messages{ messages: [ %Message.Singular{ msgid: ["hello worlds!"], references: [{"new_file.txt", 2}] } ] } assert {%Messages{messages: [message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert Message.has_flag?(message, "fuzzy") assert message.msgid == ["hello worlds!"] assert message.msgstr == ["foo"] assert message.comments == ["# old comment"] assert message.references == [{"new_file.txt", 2}] end test "simple messages can be a fuzzy match for plurals" do old_po = %Messages{ messages: [ %Message.Singular{ msgid: ["Here are {count} cocoa balls."], msgstr: ["Hier sind {count} Kakaokugeln."], comments: ["# Guyanese Cocoballs"], references: [{"old_file.txt", 1}] } ] } new_pot = %Messages{ messages: [ %Message.Plural{ msgid: ["Here is a cocoa ball."], msgid_plural: ["Here are {count} cocoa balls."], references: [{"new_file.txt", 2}] } ] } assert {%Messages{messages: [message]}, stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert Message.has_flag?(message, "fuzzy") assert message.msgid == ["Here is a cocoa ball."] assert message.msgid_plural == ["Here are {count} cocoa balls."] assert message.msgstr[0] == ["Hier sind {count} Kakaokugeln."] assert message.comments == ["# Guyanese Cocoballs"] assert message.references == [{"new_file.txt", 2}] assert %{exact_matches: 0, fuzzy_matches: 1, new: 0, removed: 0} = stats end # This has been verified with msgmerge too. test "messages fuzzy-match regardless of msgctxt" do old_po = %Messages{ messages: [ %Message.Singular{msgid: "hello world!", msgctxt: "context", msgstr: ["cfoo"]} ] } new_pot = %Messages{ messages: [ %Message.Singular{ msgid: "hello worlds!", msgctxt: "completely different" }, %Message.Singular{msgid: "different", msgctxt: "context"} ] } assert {%Messages{messages: [fuzzy_message, new_message]}, stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert %{exact_matches: 0, fuzzy_matches: 1, new: 1, removed: 0} = stats assert fuzzy_message.msgid == "hello worlds!" assert fuzzy_message.msgstr == ["cfoo"] assert fuzzy_message.msgctxt == "completely different" assert fuzzy_message.flags == [["fuzzy"]] assert new_message.msgid == "different" assert new_message.msgctxt == "context" end test "if there's a Plural-Forms header, it's used to determine number of plural forms" do old_po = %Messages{ headers: [~s(Plural-Forms: nplurals=3;plural=n>1;)], messages: [] } new_pot = %Messages{ messages: [ %Message.Singular{msgid: "a"}, %Message.Plural{msgid: "b", msgid_plural: "bs"} ] } assert {%Messages{messages: [message, plural_message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.msgid == "a" assert plural_message.msgid == "b" assert plural_message.msgid_plural == "bs" assert plural_message.msgstr == %{0 => [""], 1 => [""], 2 => [""]} end test "if there's a Plural-Forms header with only nplurals=, it's used but deprecated" do old_po = %Messages{ headers: [~s(Plural-Forms: nplurals=3)], messages: [] } new_pot = %Messages{ messages: [ %Message.Singular{msgid: "a"}, %Message.Plural{msgid: "b", msgid_plural: "bs"} ] } stderr = capture_io(:stderr, fn -> assert {%Messages{messages: [message, plural_message]}, _stats} = Merger.merge(old_po, new_pot, "en", @opts, @gettext_config) assert message.msgid == "a" assert plural_message.msgid == "b" assert plural_message.msgid_plural == "bs" assert plural_message.msgstr == %{0 => [""], 1 => [""], 2 => [""]} end) assert stderr =~ ~s(Plural-Forms headers in the form "nplurals=") end test "custom flags defined by :custom_flag_to_keep config are kept" do old_po = %Messages{ messages: [ %Message.Singular{ msgid: "a", flags: [["elixir-format", "fuzzy", "custom-flag", "other-custom-flag"]] } ] } new_po = %Messages{ messages: [ %Message.Singular{ msgid: "a", flags: [["elixir-format"]] } ] } gettext_config = [custom_flags_to_keep: ["custom-flag"]] {merged_message, _stats} = Merger.merge(old_po, new_po, "en", @opts, gettext_config) assert %Messages{ messages: [ %Message.Singular{flags: [["elixir-format", "fuzzy", "custom-flag"]]} ] } = merged_message end end describe "prune_references/2" do test "prunes all references when `write_reference_comments` is `false`" do po = %Messages{ messages: [ %Message.Singular{msgid: "a", references: [[{"path/to/file.ex", 12}]]}, %Message.Plural{msgid: "a", msgid_plural: "ab", references: [[{"path/to/file.ex", 12}]]} ] } config = [write_reference_comments: false] assert %Messages{ messages: [ %Message.Singular{references: []}, %Message.Plural{references: []} ] } = Merger.prune_references(po, config) end test "prunes reference line numbers when `write_reference_line_numbers` is `false`" do po = %Messages{ messages: [ %Message.Singular{ msgid: "a", references: [ [{"path/to/file.ex", 12}, {"path/to/file.ex", 24}, {"a", 1}], [{"path/to/file.ex", 42}, {"b", 1}], [{"path/to/file.ex", 42}], [{"path/to/other_file.ex", 24}] ] }, %Message.Plural{msgid: "a", msgid_plural: "ab", references: [[{"path/to/file.ex", 12}]]} ] } config = [write_reference_line_numbers: false] assert %Messages{ messages: [ %Message.Singular{ references: [ ["path/to/file.ex", "a"], ["b"], ["path/to/other_file.ex"] ] }, %Message.Plural{references: [["path/to/file.ex"]]} ] } = Merger.prune_references(po, config) end test "does nothing per default" do po = %Messages{ messages: [ %Message.Singular{msgid: "a", references: [[{"path/to/file.ex", 12}]]}, %Message.Plural{msgid: "a", msgid_plural: "ab", references: [{"path/to/file.ex", 12}]} ] } config = [] assert po == Merger.prune_references(po, config) end end @tag :tmp_dir test "new_po_file/2", %{tmp_dir: tmp_dir} do pot_path = Path.join(tmp_dir, "new_po_file.pot") new_po_path = Path.join(tmp_dir, "it/LC_MESSAGES/new_po_file.po") write_file(pot_path, """ ## Stripme! # A comment msgid "foo" msgstr "bar" msgid "plural" msgid_plural "plurals" msgstr[0] "" msgstr[1] "" msgctxt "my_context" msgid "with context" msgstr "" """) {new_po, _stats} = Merger.new_po_file(new_po_path, pot_path, "it", [plural_forms: 1] ++ @opts) assert new_po.file == new_po_path assert new_po.headers == ["", "Language: it\n", "Plural-Forms: nplurals=1\n"] assert ["# \"msgid\"s in this file come from POT (.pot) files.", "##" | _] = new_po.top_comments assert [ %Message.Singular{} = message, %Message.Plural{} = plural_message, %Message.Singular{} = context_message ] = new_po.messages assert message.comments == [" A comment"] assert message.msgid == ["foo"] assert message.msgstr == ["bar"] assert plural_message.msgid == ["plural"] assert plural_message.msgid_plural == ["plurals"] assert plural_message.msgstr == %{0 => [""]} assert context_message.msgctxt == ["my_context"] assert context_message.msgid == ["with context"] end defp write_file(path, contents) do path |> Path.dirname() |> File.mkdir_p!() File.write!(path, contents) end end ================================================ FILE: test/gettext/new_backend_setup_test.exs ================================================ # https://github.com/elixir-gettext/gettext/issues/330 defmodule Gettext.NewBackendSetupTest do # Has to be async: false since it changes Elixir compiler options. use ExUnit.Case, async: false @moduletag :tmp_dir defmodule Backend do use Gettext.Backend, otp_app: :test_application end describe "use Gettext, backend: ..." do test "imports Gettext.Macros macros but doesn't generate functions" do {{:module, mod, _bytecode, _funs}, _bindings} = Code.eval_quoted( quote do defmodule MyModule do use Gettext, backend: Gettext.NewBackendSetupTest.Backend def translate do gettext("Hello world") end end end ) refute function_exported?(mod, :__gettext__, 1) assert mod.translate() == "Hello world" end end describe "compile-time dependencies" do test "are not created for modules that use the backend", %{test: test} do top_level_module = :"Elixir.Gettext_#{test}" backend_module = Module.concat(top_level_module, Gettext) Code.eval_quoted( quote do defmodule unquote(backend_module) do use Gettext.Backend, otp_app: unquote(test) end end ) old_compiler_opts = Code.compiler_options(tracers: [__MODULE__]) on_exit(fn -> Code.compiler_options(old_compiler_opts) end) Code.compile_quoted( quote do defmodule unquote(top_level_module) do use Gettext, backend: unquote(backend_module) end end ) refute_received {:trace, {:require, _meta, ^backend_module, _opts}} refute_received {:trace, {:import, _meta, ^backend_module, _opts}} end end def trace(event, _env) do send(self(), {:trace, event}) :ok end end ================================================ FILE: test/gettext/plural_test.exs ================================================ defmodule Gettext.PluralTest do use ExUnit.Case, async: true import Gettext.Plural, only: [nplurals: 1, plural: 2] alias Gettext.Plural.UnknownLocaleError doctest Gettext.Plural test "x_* locales are pluralized like x except for exceptions" do assert nplurals("en") == nplurals("en_GB") assert plural("pt", 0) == 1 assert plural("pt", 1) == 0 assert plural("pt_BR", 0) == 0 assert plural("pt_BR", 1) == 0 end test "locale with a territory" do # The _XX in en_XX gets stripped and en_XX is pluralized as en. assert nplurals("en_XX") == nplurals("en") assert plural("en_XX", 100) == plural("en", 100) end test "unknown locale" do message = ~r/unknown locale "wat"/ assert_raise UnknownLocaleError, message, fn -> nplurals("wat") end assert_raise UnknownLocaleError, message, fn -> plural("wat", 1) end # This happens with dash as the territory/locale separator # (https://en.wikipedia.org/wiki/IETF_language_tag). message = ~r/unknown locale "en-us"/ assert_raise UnknownLocaleError, message, fn -> nplurals("en-us") end end test "locales with one form" do assert nplurals("ja") == 1 assert plural("ja", 0) == 0 assert plural("ja", 8) == 0 end test "locales with two forms where 0 is same as > 1" do assert nplurals("it") == 2 assert plural("it", 1) == 0 assert plural("it", 0) == 1 assert plural("it", 13) == 1 end test "locales with two forms where 0 and 1 are the same" do assert nplurals("fr") == 2 assert plural("fr", 0) == 0 assert plural("fr", 1) == 0 assert plural("fr", 2) == 1 end test "locales that belong to the 3-forms slavic family" do assert nplurals("ru") == 3 assert plural("ru", 21) == 0 assert plural("ru", 42) == 1 assert plural("ru", 11) == 2 end test "locales that belong to the alternative 3-forms slavic family" do assert nplurals("cs") == 3 assert plural("cs", 1) == 0 assert plural("cs", 3) == 1 assert plural("cs", 12) == 2 end test "locales that don't belong to any pluralization family" do assert plural("ar", 0) == 0 assert plural("ar", 1) == 1 assert plural("ar", 2) == 2 assert plural("ar", 505) == 3 assert plural("ar", 733) == 4 assert plural("ar", 101) == 5 assert plural("csb", 1) == 0 assert plural("csb", 33) == 1 assert plural("csb", 115) == 2 assert plural("cy", 1) == 0 assert plural("cy", 2) == 1 assert plural("cy", 23) == 2 assert plural("cy", 8) == 3 assert plural("ga", 1) == 0 assert plural("ga", 2) == 1 assert plural("ga", 4) == 2 assert plural("ga", 10) == 3 assert plural("ga", 133) == 4 assert plural("gd", 1) == 0 assert plural("gd", 12) == 1 assert plural("gd", 18) == 2 assert plural("gd", 20) == 3 assert plural("is", 71) == 0 assert plural("is", 11) == 1 assert plural("jv", 0) == 0 assert plural("jv", 13) == 1 assert plural("kw", 1) == 0 assert plural("kw", 2) == 1 assert plural("kw", 3) == 2 assert plural("kw", 99) == 3 assert plural("lt", 81) == 0 assert plural("lt", 872) == 1 assert plural("lt", 112) == 2 assert plural("lv", 31) == 0 assert plural("lv", 9) == 1 assert plural("lv", 0) == 2 assert plural("mk", 131) == 0 assert plural("mk", 132) == 1 assert plural("mk", 9) == 2 assert plural("mnk", 0) == 0 assert plural("mnk", 1) == 1 assert plural("mnk", 12) == 2 assert plural("mt", 1) == 0 assert plural("mt", 0) == 1 assert plural("mt", 119) == 2 assert plural("mt", 67) == 3 assert plural("pl", 1) == 0 assert plural("pl", 102) == 1 assert plural("pl", 713) == 2 assert plural("ro", 1) == 0 assert plural("ro", 19) == 1 assert plural("ro", 80) == 2 assert plural("sl", 320) == 0 assert plural("sl", 101) == 1 assert plural("sl", 202) == 2 assert plural("sl", 303) == 3 end end ================================================ FILE: test/gettext_test.exs ================================================ defmodule GettextTest.TranslatorWithDuckInterpolator.Interpolator do @behaviour Gettext.Interpolation @impl Gettext.Interpolation def runtime_interpolate(message, bindings), do: {:ok, "quack #{message} #{inspect(bindings)} quack"} @impl Gettext.Interpolation defmacro compile_interpolate(_message_type, message, bindings) do quote do {:ok, "quack #{unquote(message)} #{inspect(unquote(bindings))} quack"} end end @impl Gettext.Interpolation def message_format, do: "duck-format" end defmodule GettextTest.TranslatorWithDuckInterpolator do use Gettext.Backend, otp_app: :test_application, interpolation: GettextTest.TranslatorWithDuckInterpolator.Interpolator, priv: "test/fixtures/single_messages" end defmodule GettextTest do use ExUnit.Case import ExUnit.CaptureIO import ExUnit.CaptureLog alias GettextTest.Backend describe "get_locale/0,1" do test "returns \"en\" as the default" do assert Gettext.get_locale() == "en" assert Gettext.get_locale(Backend) == "en" end test "gets the locale set by put_locale/2" do # First, we set the local for just one backend: Gettext.put_locale(Backend, "pt_BR") # Now, let's check that only that backend was affected. assert Gettext.get_locale(Backend) == "pt_BR" assert Gettext.get_locale() == "en" # Now, let's change the global locale: Gettext.put_locale("it") # Let's check that the global locale was affected and that get_locale/1 # returns the global locale, but only for backends that have no # backend-specific locale set. assert Gettext.get_locale() == "it" assert Gettext.get_locale(Backend) == "pt_BR" end test "uses the default locale of the :gettext application" do global_default = Application.get_env(:gettext, :default_locale) try do Application.put_env(:gettext, :default_locale, "fr") assert Gettext.get_locale() == "fr" assert Gettext.get_locale(Backend) == "fr" after Application.put_env(:gettext, :default_locale, global_default) end end end describe "put_locale/2" do test "only accepts binaries" do msg = "put_locale/2 only accepts binary locales, got: :en" assert_raise ArgumentError, msg, fn -> Gettext.put_locale(Backend, :en) end end end describe "put_locale!/2" do test "raises an error when unsupported locale is passed as argument" do msg = "put_locale!/2 only support known locales, got: \"kr\"" assert Gettext.known_locales(Backend) == ["it", "ja"] assert_raise ArgumentError, msg, fn -> Gettext.put_locale!(Backend, "kr") end end end describe "with_locale/3" do test "doesn't raise if no locale was set (defaulting to 'en')" do Process.delete(Backend) Gettext.with_locale(Backend, "it", fn -> assert Gettext.gettext(Backend, "Hello world") == "Ciao mondo" end) assert Gettext.get_locale(Backend) == "en" end test "runs a function with a given locale and returns the returned value" do Gettext.put_locale(Backend, "fr") # no 'fr' message assert Gettext.gettext(Backend, "Hello world") == "Hello world" res = Gettext.with_locale(Backend, "it", fn -> assert Gettext.gettext(Backend, "Hello world") == "Ciao mondo" :foo end) assert Gettext.get_locale(Backend) == "fr" assert res == :foo end test "resets the locale even if the given function raises" do Gettext.put_locale(Backend, "fr") assert_raise RuntimeError, fn -> Gettext.with_locale(Backend, "it", fn -> raise "foo" end) end assert Gettext.get_locale(Backend) == "fr" catch_throw(Gettext.with_locale(Backend, "it", fn -> throw(:foo) end)) assert Gettext.get_locale(Backend) == "fr" end end describe "known_locales/1" do test "returns all the locales for which a backend has PO files" do assert Gettext.known_locales(Backend) == ["it", "ja"] assert Gettext.known_locales(GettextTest.BackendWithAllowedLocalesAtom) == ["es"] assert Gettext.known_locales(GettextTest.BackendWithAllowedLocalesString) == ["es"] end end test "a custom default_domain can be set for a backend" do Code.eval_quoted( quote do defmodule DefaultDomainTest do use Gettext, backend: GettextTest.BackendWithDefaultDomain def test("Invalid email address"), do: gettext("Invalid email address") def test("Hello world"), do: gettext("Hello world") end end ) Gettext.put_locale("it") assert apply(DefaultDomainTest, :test, ["Invalid email address"]) == "Indirizzo email non valido" assert apply(DefaultDomainTest, :test, ["Hello world"]) == "Hello world" end test "MissingBindingsError log messages" do assert capture_log(fn -> Gettext.pgettext(Backend, "test", "Hello %{name}, missing message!", %{}) end) =~ "missing Gettext bindings: [:name] (backend GettextTest.Backend," <> " locale \"en\", domain \"default\", msgctxt \"test\", msgid \"Hello " <> "%{name}, missing message!\")" end describe "*gettext functions (singular)" do setup do Gettext.put_locale(Backend, "it") :ok end test "gettext/2" do assert Gettext.gettext(Backend, "Hello world") == "Ciao mondo" assert Gettext.gettext(Backend, "Nonexistent") == "Nonexistent" end test "dgettext/4" do msgid = "Invalid email address" assert Gettext.dgettext(Backend, "errors", msgid) == "Indirizzo email non valido" assert Gettext.dgettext(Backend, "foo", "Foo") == "Foo" log = capture_log(fn -> assert Gettext.dgettext(Backend, "interpolations", "Hello %{name}", %{}) == "Ciao %{name}" end) assert log =~ "[error] missing Gettext bindings: [:name]" end test "pgettext/3" do assert Gettext.pgettext(Backend, "test", "Hello world") == "Ciao mondo" assert Gettext.pgettext(Backend, "test", "Nonexistent") == "Nonexistent" end test "dpgettext/4" do assert Gettext.dpgettext(Backend, "default", "test", "Hello world") == "Ciao mondo" end end describe "*ngettext functions (plural)" do setup do Gettext.put_locale(Backend, "it") :ok end test "ngettext/5" do msgid = "One cake, %{name}" msgid_plural = "%{count} cakes, %{name}" assert Gettext.ngettext(Backend, msgid, msgid_plural, 1, %{name: "Meg"}) == "One cake, Meg" assert Gettext.ngettext(Backend, msgid, msgid_plural, 5, %{name: "Meg"}) == "5 cakes, Meg" end test "dngettext/6" do msgid = "You have one message, %{name}" msgid_plural = "You have %{count} messages, %{name}" assert Gettext.dngettext(Backend, "interpolations", msgid, msgid_plural, 1, %{name: "Meg"}) == "Hai un messaggio, Meg" assert Gettext.dngettext(Backend, "interpolations", msgid, msgid_plural, 5, %{name: "Meg"}) == "Hai 5 messaggi, Meg" assert Gettext.dngettext(Backend, "interpolations", "Month", "%{count} months", 5) == "5 mesi" end test "pngettext/6" do msgctxt = "test" msgid = "One cake, %{name}" msgid_plural = "%{count} cakes, %{name}" assert Gettext.pngettext(Backend, msgctxt, msgid, msgid_plural, 1, %{name: "Meg"}) == "One cake, Meg" assert Gettext.pngettext(Backend, msgctxt, msgid, msgid_plural, 5, %{name: "Meg"}) == "5 cakes, Meg" end test "dpngettext/6" do msgid = "You have one message, %{name}" msgid_plural = "You have %{count} messages, %{name}" assert Gettext.dpngettext(Backend, "interpolations", "test", msgid, msgid_plural, 1, %{ name: "Meg" }) == "Hai un messaggio, Meg" assert Gettext.dpngettext(Backend, "interpolations", "test", msgid, msgid_plural, 5, %{ name: "Meg" }) == "Hai 5 messaggi, Meg" assert Gettext.dpngettext( Backend, "default", "test", "One new email", "%{count} new emails", 5, %{name: "Meg"} ) == "5 nuove test email" end end test "the d?n?gettext functions support kw list for interpolations" do Gettext.put_locale(Backend, "it") assert Gettext.gettext(Backend, "Hello %{name}", name: "José") == "Hello José" end test "uses custom interpolator" do assert Gettext.gettext(GettextTest.TranslatorWithDuckInterpolator, "foo") == "quack foo %{} quack" end test "use Gettext for defining backends is deprecated" do stderr = capture_io(:stderr, fn -> Code.eval_quoted( quote do defmodule DeprecatedWayOfDefiningBackend do use Gettext, otp_app: :my_app end end ) end) expected_message = """ Defining a Gettext backend by calling: use Gettext, otp_app: :my_app is deprecated. To define a backend, call: use Gettext.Backend, otp_app: :my_app Then, replace importing your backend: import DeprecatedWayOfDefiningBackend with calling this in your module: use Gettext, backend: DeprecatedWayOfDefiningBackend nofile:1: DeprecatedWayOfDefiningBackend (module) """ assert stderr =~ expected_message end end ================================================ FILE: test/mix/tasks/gettext.extract_test.exs ================================================ defmodule Mix.Tasks.Gettext.ExtractTest do use ExUnit.Case import ExUnit.CaptureIO import GettextTest.MixProjectHelpers @moduletag :tmp_dir setup_all do # To suppress the `redefining module MyApp` warnings for the test modules Code.compiler_options(ignore_module_conflict: true) :ok end test "extracting and extracting with --merge", %{test: test, tmp_dir: tmp_dir} = context do create_test_mix_file(context) write_file(context, "lib/my_app.ex", """ defmodule MyApp.Gettext do use Gettext.Backend, otp_app: #{inspect(test)} end defmodule MyApp do use Gettext, backend: MyApp.Gettext def foo(), do: gettext("hello") end """) output = capture_io(fn -> in_project(test, tmp_dir, fn _module -> run([]) end) end) assert output =~ "Extracted priv/gettext/default.pot" assert read_file(context, "priv/gettext/default.pot") =~ """ #: lib/my_app.ex:7 #, elixir-autogen, elixir-format msgid "hello" msgstr "" """ # Test --merge too. write_file(context, "lib/other.ex", """ defmodule MyApp.Other do use Gettext, backend: MyApp.Gettext def foo(), do: dgettext("my_domain", "other") end """) write_file(context, "priv/gettext/it/LC_MESSAGES/my_domain.po", "") capture_io(fn -> in_project(test, tmp_dir, fn _module -> run(["--merge"]) end) end) assert read_file(context, "priv/gettext/it/LC_MESSAGES/my_domain.po") == """ #: lib/other.ex:3 #, elixir-autogen, elixir-format msgid "other" msgstr "" """ capture_io(fn -> in_project(test, tmp_dir, fn _module -> run(["--merge"]) end) end) =~ "Wrote priv/gettext/it/LC_MESSAGES/my_domain.po" end test "--check-up-to-date should fail if no POT files have been created", %{test: test, tmp_dir: tmp_dir} = context do create_test_mix_file(context) write_file(context, "lib/my_app.ex", """ defmodule MyApp.Gettext do use Gettext.Backend, otp_app: #{inspect(test)} end defmodule MyApp do use Gettext, backend: MyApp.Gettext def foo(), do: gettext("hello") end """) write_file(context, "lib/other.ex", """ defmodule MyApp.Other do use Gettext, backend: MyApp.Gettext def foo(), do: dgettext("my_domain", "other") end """) expected_message = """ mix gettext.extract failed due to --check-up-to-date. The following POT files were not extracted or are out of date: * priv/gettext/default.pot * priv/gettext/my_domain.pot """ capture_io(fn -> assert_raise Mix.Error, expected_message, fn -> in_project(test, tmp_dir, fn _module -> run(["--check-up-to-date"]) end) end end) end test "--check-up-to-date should pass if nothing changed", %{test: test, tmp_dir: tmp_dir} = context do create_test_mix_file(context, write_reference_comments: false) write_file(context, "lib/my_app.ex", """ defmodule MyApp.Gettext do use Gettext.Backend, otp_app: #{inspect(test)} end defmodule MyApp do use Gettext, backend: MyApp.Gettext def foo(), do: gettext("hello") end """) capture_io(fn -> in_project(test, tmp_dir, fn _module -> run([]) end) in_project(test, tmp_dir, fn _module -> run(["--check-up-to-date"]) end) end) end test "--check-up-to-date should fail if POT files are outdated", %{test: test, tmp_dir: tmp_dir} = context do create_test_mix_file(context) write_file(context, "lib/my_app.ex", """ defmodule MyApp.Gettext do use Gettext.Backend, otp_app: #{inspect(test)} end defmodule MyApp do use Gettext, backend: MyApp.Gettext def foo(), do: gettext("hello") end """) write_file(context, "lib/other.ex", """ defmodule MyApp.Other do use Gettext, backend: MyApp.Gettext def foo(), do: dgettext("my_domain", "other") end """) capture_io(fn -> in_project(test, tmp_dir, fn _module -> run([]) end) end) write_file(context, "lib/my_app.ex", """ defmodule MyApp.Gettext do use Gettext.Backend, otp_app: #{inspect(test)} end defmodule MyApp do use Gettext, backend: MyApp.Gettext def foo(), do: gettext("hello world") end """) expected_message = """ mix gettext.extract failed due to --check-up-to-date. The following POT files were not extracted or are out of date: * priv/gettext/default.pot """ capture_io(fn -> assert_raise Mix.Error, expected_message, fn -> in_project(test, tmp_dir, fn _module -> run(["--check-up-to-date"]) end) end end) end defp run(args) do Mix.Tasks.Gettext.Extract.run(args) end end ================================================ FILE: test/mix/tasks/gettext.merge_test.exs ================================================ defmodule Mix.Tasks.Gettext.MergeTest do use ExUnit.Case import ExUnit.CaptureIO test "raises an error when if one of the files doesn't exist" do assert_raise Mix.Error, "No such file: foo.po", fn -> run(~w(foo.po bar.pot)) end end test "raises an error if the files aren't a .po file and a .pot file" do assert_raise Mix.Error, "Arguments must be a PO file and a PO/POT file", fn -> run(~w(foo.ex bar.exs)) end end test "passing more than one argument raises an error" do assert_raise Mix.Error, ~r/^You can only pass one or two arguments/, fn -> run(~w(foo bar baz bong)) end end test "passing no arguments raises an error" do assert_raise Mix.Error, ~r/You can only pass one or two arguments/, fn -> run([]) end end @tag :tmp_dir test "passing a :fuzzy_threshold outside of 0..1 raises an error", %{tmp_dir: tmp_dir} do File.mkdir_p(tmp_dir) assert_raise Mix.Error, "The :fuzzy_threshold option must be a float >= 0.0 and <= 1.0", fn -> run([tmp_dir, "--fuzzy-threshold", "5.0"]) end end @tag :tmp_dir test "merging an existing PO file with a new POT file", %{tmp_dir: tmp_dir} do pot_contents = """ msgid "hello" msgstr "" """ write_file(Path.join(tmp_dir, "foo.pot"), pot_contents) write_file(Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), "") output = capture_io(fn -> run([Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), Path.join(tmp_dir, "foo.pot")]) end) assert output =~ ~r{Wrote .*/it/LC_MESSAGES/foo\.po} assert output =~ "(1 new message, 0 removed, 0 unchanged, 0 reworded (fuzzy), 0 marked as obsolete)" # The POT file is left unchanged assert File.read!(Path.join(tmp_dir, "foo.pot")) == pot_contents assert File.read!(Path.join(tmp_dir, "it/LC_MESSAGES/foo.po")) == """ msgid "hello" msgstr "" """ end @tag :tmp_dir test "marks messages as obsolete", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "foo.pot"), "") write_file(Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), """ msgid "foo" msgstr "" """) output = capture_io(fn -> run([ Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), Path.join(tmp_dir, "foo.pot"), "--on-obsolete", "mark_as_obsolete" ]) end) assert output =~ ~r{Wrote .*/it/LC_MESSAGES/foo.po} assert output =~ "(0 new messages, 0 removed, 0 unchanged, 0 reworded (fuzzy), 1 marked as obsolete)" end @tag :tmp_dir test "removes obsolete messages", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "foo.pot"), "") write_file(Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), """ msgid "foo" msgstr "" """) output = capture_io(fn -> run([ Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), Path.join(tmp_dir, "foo.pot"), "--on-obsolete", "delete" ]) end) assert output =~ ~r{Wrote .*/it/LC_MESSAGES/foo.po} assert output =~ "(0 new messages, 1 removed, 0 unchanged, 0 reworded (fuzzy), 0 marked as obsolete)" end @tag :tmp_dir test "validates on-obsolete", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "foo.pot"), "") write_file(Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), "") expected_message = """ An invalid value was provided for the option `on_obsolete`. Value: "invalid" Valid Choices: "delete" / "mark_as_obsolete" """ assert_raise Mix.Error, expected_message, fn -> run([ Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), Path.join(tmp_dir, "foo.pot"), "--on-obsolete", "invalid" ]) end end @tag :tmp_dir test "passing a dir and a --locale opt will update/create PO files in the locale dir", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "default.pot"), """ msgid "def" msgstr "" """) write_file(Path.join(tmp_dir, "new.pot"), """ msgid "new" msgstr "" """) write_file(Path.join(tmp_dir, "it/LC_MESSAGES/default.po"), "") output = capture_io(fn -> run([tmp_dir, "--locale", "it"]) end) assert output =~ ~r{Wrote .*/it/LC_MESSAGES/new.po} assert output =~ ~r{Wrote .*/it/LC_MESSAGES/default.po} assert File.read!(Path.join(tmp_dir, "it/LC_MESSAGES/default.po")) == """ msgid "def" msgstr "" """ new_po = File.read!(Path.join(tmp_dir, "it/LC_MESSAGES/new.po")) assert new_po =~ ~S""" msgid "" msgstr "" "Language: it\n" "Plural-Forms: nplurals=2; plural=(n != 1);\n" msgid "new" msgstr "" """ assert String.starts_with?(new_po, "## \"msgid\"s in this file come from POT") end @tag :tmp_dir test "enabling --store-previous-message-on-fuzzy-match stores previous message", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "default.pot"), """ msgid "Hello Worlds" msgstr "" """) write_file(Path.join(tmp_dir, "it/LC_MESSAGES/default.po"), """ msgid "Hello World" msgstr "" """) output = capture_io(fn -> run([tmp_dir, "--locale", "it", "--store-previous-message-on-fuzzy-match"]) end) assert output =~ ~r{Wrote .*/it/LC_MESSAGES/default.po} assert File.read!(Path.join(tmp_dir, "it/LC_MESSAGES/default.po")) == """ #, fuzzy #| msgid "Hello World" msgid "Hello Worlds" msgstr "" """ end @tag :tmp_dir test "passing a dir and a --locale opt will update/create PO files in the locale dir with custom plural forms", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "new.pot"), """ msgid "new" msgstr "" """) output = capture_io(fn -> run([ tmp_dir, "--locale", "it", "--plural-forms-header", "nplurals=3; plural=n==0 ? 0 : n > 1;" ]) end) assert output =~ ~r{Wrote .*/it/LC_MESSAGES/new.po} assert File.read!(Path.join(tmp_dir, "it/LC_MESSAGES/new.po")) =~ ~S""" msgid "" msgstr "" "Language: it\n" "Plural-Forms: nplurals=3; plural=n==0 ? 0 : n > 1;\n" msgid "new" msgstr "" """ end @tag :tmp_dir test "passing a dir and a --locale opt will update/create PO files in the locale dir with app env plural forms", %{tmp_dir: tmp_dir} do Application.put_env(:gettext, :plural_forms, GettextTest.CustomPlural) write_file(Path.join(tmp_dir, "new.pot"), """ msgid "new" msgstr "" """) output = capture_io(fn -> run([tmp_dir, "--locale", "elv"]) end) assert output =~ ~r{Wrote .*/elv/LC_MESSAGES/new.po} assert File.read!(Path.join(tmp_dir, "elv/LC_MESSAGES/new.po")) =~ ~S""" msgid "" msgstr "" "Language: elv\n" "Plural-Forms: nplurals=2\n" msgid "new" msgstr "" """ after Application.put_env(:gettext, :plural_forms, Gettext.Plural) end @tag :tmp_dir test "passing just a dir merges with PO files in every locale", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "fr/LC_MESSAGES/foo.po"), "") write_file(Path.join(tmp_dir, "it/LC_MESSAGES/foo.po"), "") contents = """ msgid "foo" msgstr "" """ write_file(Path.join(tmp_dir, "foo.pot"), contents) output = capture_io(fn -> run([tmp_dir]) end) assert output =~ ~r{Wrote .*/fr/LC_MESSAGES/foo.po} assert output =~ ~r{Wrote .*/it/LC_MESSAGES/foo.po} assert File.read!(Path.join(tmp_dir, "fr/LC_MESSAGES/foo.po")) == contents assert File.read!(Path.join(tmp_dir, "it/LC_MESSAGES/foo.po")) == contents end @tag :tmp_dir test "non-existing locale/LC_MESSAGES directories are created", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "foo.pot"), """ msgid "foo" msgstr "" """) created_dir = Path.join([tmp_dir, "en", "LC_MESSAGES"]) refute File.dir?(created_dir) output = capture_io(fn -> run([tmp_dir, "--locale", "en"]) end) assert File.dir?(created_dir) assert output =~ "Created directory #{created_dir}" end @tag :tmp_dir test "informative comments at the top of the file", %{tmp_dir: tmp_dir} do write_file(Path.join(tmp_dir, "inf.pot"), """ msgid "foo" msgstr "" """) capture_io(:stdio, fn -> capture_io(:stderr, fn -> run([tmp_dir, "--locale", "en"]) contents = File.read!(Path.join(tmp_dir, "en/LC_MESSAGES/inf.po")) assert contents =~ "## \"msgid\"s in this file" # Running the task again without having change the PO file shouldn't # remove the informative comment. run([tmp_dir, "--locale", "en"]) assert contents == File.read!(Path.join(tmp_dir, "en/LC_MESSAGES/inf.po")) end) end) end defp write_file(path, contents) do File.mkdir_p!(Path.dirname(path)) File.write!(path, contents) end defp run(args) do Mix.Tasks.Gettext.Merge.run(args) end end ================================================ FILE: test/support/mix_project_helpers.ex ================================================ defmodule GettextTest.MixProjectHelpers do def create_test_mix_file(context, gettext_config \\ []) do write_file(context, "mix.exs", """ defmodule MyApp.MixProject do use Mix.Project def project() do [app: #{inspect(context.test)}, version: "0.1.0", gettext: #{inspect(gettext_config)}] end def application() do [extra_applications: [:logger, :gettext]] end end """) end def write_file(context, path, contents) do path = Path.join(context.tmp_dir, path) File.mkdir_p!(Path.dirname(path)) File.write!(path, contents) end def read_file(context, path) do context.tmp_dir |> Path.join(path) |> File.read!() end def in_project(module, dir, fun) do Mix.Project.in_project(module, dir, [prune_code_paths: false], fun) end end ================================================ FILE: test/test_helper.exs ================================================ defmodule GettextTest.CustomPlural do @behaviour Gettext.Plural def nplurals("elv"), do: 2 def nplurals(other), do: Gettext.Plural.nplurals(other) # Opposite of Italian (where 1 is singular, everything else is plural) def plural("it", 1), do: 1 def plural("it", _), do: 0 end defmodule GettextTest.CustomCompiledPlural do @behaviour Gettext.Plural @impl Gettext.Plural def init(plural_info), do: plural_info @impl Gettext.Plural def nplurals(plural_info) do send(self(), {:nplurals_context, plural_info}) plural_info |> Gettext.Plural.init() |> Gettext.Plural.nplurals() end @impl Gettext.Plural def plural(plural_info, count) do send(self(), {:plural_context, plural_info}) plural_info |> Gettext.Plural.init() |> Gettext.Plural.plural(count) end end defmodule GettextTest.Backend do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/single_messages" def handle_missing_translation(locale, domain, msgctxt, msgid, bindings) do send(self(), {locale, domain, msgctxt, msgid, bindings}) super(locale, domain, msgctxt, msgid, bindings) end def handle_missing_plural_translation( locale, domain, msgctxt, msgid, msgid_plural, n, bindings ) do send(self(), {locale, domain, msgctxt, msgid, msgid_plural, n, bindings}) super(locale, domain, msgctxt, msgid, msgid_plural, n, bindings) end end defmodule GettextTest.BackendWithAllowedLocalesString do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/multi_messages", allowed_locales: ["es"] end defmodule GettextTest.BackendWithAllowedLocalesAtom do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/multi_messages", allowed_locales: [:es] end defmodule GettextTest.BackendWithDefaultDomain do use Gettext.Backend, otp_app: :test_application, priv: "test/fixtures/single_messages", default_domain: "errors" end ExUnit.start()